I got the problem figured out, sorta. I have narrowed it down to Ubuntu 7.04. I didn't think of it until now, but I moved Vista over to my new bigger hard drive (the Seagate 160gb 7200rpm, it's awesome) and then I'm trying Ubuntu 7.04 on my old 100gb drive, and right at this time the problem must have appeared. I think the main issue was that I did not notice the wrong clock because I always wear my watch so I don't look at the computer clock very often, so it seemed to move ahead at random times, whereas if it were something more immediately noticeable then I might have linked it to the fact that it happens after I switch back to Windows from Linux.
However I'm still working out the fix for it... Linux is a pain to set up but once it's all working THEN you can have fun with customization and maybe actually get something done... But Windows will always be my primary operating system. I believe in some setup of the computer, especially adding programs that you want and getting everything working right, but to spend days or even weeks just getting your hardware completely working is insane, and Linux is not to the point yet where it is as easy to set up and get working as Windows. Of course it also depends on the system, and my Dell 1500 wireless adapter does not help the cause... There is still no working native driver so I end up using ndiswrapper so that I can just use the Windows driver on Linux and I really think that's a hacky method but it works... Most of the time. Still, if Linux were just a bit more plug-and-play, and if it had the programs that I NEED, then I might switch over. These are programs such as Visual Studio 2005 and iTunes - and stuff I want too, like Supreme Commander, Oblivion (yes i know it runs in wine), Total Annihilation (haven't tried it in wine), etc. Especially Visual Studio though. I do not know of a solution where I can create and run ASP.NET applications (with VB.NET backends) inside Linux, and my job kinda requires it so I literally CAN'T switch to Linux, otherwise I might be giving it more attention than I have been.
Still, it's fun to play around with Linux. I'll work on this clock issue in my spare time, it's just a matter of begging the geeks at #ubuntu on Freenode to answer my questions and give me step-by-step instructions