Well, you'd have to choose between speed and stability. If you install Win2K/XP, you'll have a stable machine, but it'll run more slowly. If you get rid of WinME and install 98, it'll be less stable but will run faster.
About the usb: I run a usb2 flash drive on a 1.1 port on a regular basis, and it works fine in Win2k, 98, and Linux. It sounds like you have a driver problem there. Check online for drivers for your flashdrive, or install Win2k or xp (they include the driver).
One of the best ways you could speed it up would be putting in a faster hard drive. If you just get a relatively cheap 7800rpm hard drive (40-80 Gb should be about $50 or less on Newegg) it will make a big difference, especially considering you probably have a 5400 ATA-66 in there now. Adding ram helps, too, up to a certain point, but it sounds like you've got plenty of ram for what you're using the machine for.