There is no Problem booting Linux from an USB-Stick (mine is from Lexar). Just plug it in and check in the BIOS in the 'Boot'-Section that the Stick shows up and change - if necessary - the order of the devices so that it is listed before the Harddisk.
If your USB device doesn't boot make sure you have the latest BIOS downloaded (eg from http://www.flybook.biz/ and installed. With my original BIOS version I could boot only from one really old USB-Stick and from an USB floppy drive (very useful for upgrading the BIOS...).
If you have a CD/DVD-drive with USB-interface instead it is of course easier to install the OS directly from CD/DVD and not mess around with several Linux installations on different partitions, though it is a nice challenge and there is much to learn.
One remark - the first and the last - regarding windows. When booting it for the first time you get immediately to the setup routine, with horrible Sound & Music playing. Unfortunately the volume cannot be muted or at least decreased. So I booted it exactly once, switched it off very quickly and then repartitioned the harddisk.
Boot Feather-Linux or the installer of your preferred distribution. I used
Feather-Linux as an intermediate step because the driver for the IDE-Chipset
(Ali 15x3) was not included in the ramdisk of Debian Sarge (and thus the
harddisk was not detected) and I didn't want to mess around with the
From Feather-Linux install knoppix to a separate partition and from there continue following the description at http://www.debian.org/releases/stable/i386/ch-preparing.en.html#s-linux-upgrade (link doesn't work any more, somewhere on debian.org there will be the necessary documentation).
Finally you have a pure clean Debian-System distributed on several partitions.
The easy way is to install Linux from a running Windows System, a nice document about that can be found at http://marc.herbert.free.fr/linux/win2linstall.html