But it was a pain to administer them remotely. I would start up VMware on my WIndows box here at home and load Ubuntu as a virtual machine just to use the Terminal to use SSH. Way too much involved for using the command line if you ask me.
So, I Googled a bit and found Cygwin/X. Cygwin/X is a port of the X Window System to Win32, so theoretically, you can run Linux-designed programs in Windows (not natively, they need to be recompiled especially for this). I don't need all that. I just want a bash terminal in Windows, and I got it:

So, now I can run any bash command on these machines remotely from my Windows machine!