The last 1/3 of the messages here are very informative about this problem:
The SP3 installer lies. When it says "performing cleanup," that's probably not what it's doing. It's probably installing hotfixes!
Run event viewer and look at the system events, to see what it has been up to. You'll probably see a bunch of hotfixes being installed.
In my case, the hotfixes were being installed at a glacial rate: one every 10-15 minutes. Process Explorer revealed two invisible instances of Internet Explorer, one of which was hogging the CPU. After I killed Internet Explorer, Event Viewer showed hotfixes taking about 2 minutes each, which still seems slow, but is a huge improvement.
Microsoft people, if you ever read this, THIS IS A BUG! Installing additional hotfixes is NOT "performing cleanup." When you display a message that says it is performing cleanup when it is ACTUALLY doing something else, that is FALSE information conveyed to the user, which is a BUG. SO FIX IT!!!
In this case, it is a bug with real, significant consequences, because it leads frustrated users to halt the installation when they really should let it continue to run. What it SHOULD do is display a message that says something like "installing hotfix 1 of 100" or "installing hotfix KB123456" etc.. It is IMPORTANT that users be able to tell the difference between stuck/hung and actually doing something useful. When you display the same "performing cleanup" message for hours, it is natural to assume that it is stuck/hung, so it is natural for the frustrated user to do the wrong thing (i.e., abort it -- while cursing Bill Gates' name and vowing to buy a Mac next time).