The only suggestion I have for now, til I see your answers to the questions below, would be to run a speedtest on your line, to see if the fault lies not with you, but with your Internet Service Provider. Try going to http://www.speedtest.net/
and see what your connection speed looks like, then try calling your ISP and asking them what your connection speed should be. It is possible they have a bottleneck of some kind in their trunk lines, which can happen pretty quickly, depending on the population in your area. I know I had that problem for a while and it took multiple calls (and about 6 months) for them to fix their trunk lines and get the connection 'back up to speed', so to speak.
Just curious, how do you go about disconnecting/reconnecting the cable from the computer? Are either of them powered off before disconnecting the cable? If you can, tell me your step-by-step procedures for disconnecting and reconnecting the cables.
Also, I saw you said "The only thing I can remember doing was joining xBox live around the time it started going slow." Does this mean that is also when you started switching the cables back and forth between your PC and the Xbox? And have you noticed any problems connecting to Xbox live, or with the speed while connected?