In my recent science fair that won at my county and 3rd in the state, an aspect I studied was how well four free anti-spyware programs did: Windows Defender, Spybot S&D, Ad-Aware SE, and A-Squared Anti-Malware.
Here are the results of my experiment:
A-Squared: 16 Cookies, 11 Registry, 6 Files - 68% of total amount
Ad-Aware SE: 21 Cookies, 3 Registry, 1 File - 50%
Spybot S&D: 15 Cookies, 1 Registry, 1 File - 34%
Windows Defender: 1 Registry, 1 File - 4%

All of these programs were fully updated and then brought into safe mode to do the scanning. Just thought I'd post this little FYI for people who have Windows Defender as their only anti-spyware program.

I would be interested in understanding what these four applications were tested against as well as the methodology you employed in your experiment. From the percentage, I would guess something around 50 different instances; do you consider this an adequate sample to warrant any conclusion?
Regards,
John

I would be interested in the exact methodology as well. What was your test platform and what malware installed to test the scanners?

Also exactly what do the percentages relate to? Is that detection only or removal? These can be two seperate results as some scanners may detect items but are not able to remove them. Also if these numbers are for removal, was it verified and were detections checked for false positives?

Don't get me wrong, it's an interesting study and you're to be appluaded. And while I agree that WD should not be depended on as the only supplemental security program (anti-spyware and other antimalware other than antiviruses and firewall), I have a hard time believing that WD's results were that low. You did reinfect your test platform with the same set of malware before running WD, correct?

I would also find it interesting to make comparisions to programs' effectiveness at being proactive rather than retro-active. You might find that WD is more effective at protecting a system from getting infected in the first place since the free versions of A-squared and Ad-Aware don't offer a proactive approach. That would be a totally different experiment, but would give a better idea of the overall worth of the WD program.

BTW, Elendil, since you are a HJT Trainee, now, some aspects, such as links to dangerous websites, would be better discussed in the private forum.

This is out of 49 spyware programs. The basic summary goes as follows:
1. Updated all of the programs,
2. Went out and got my computer infected (Some GAIN, random ads, downloads (limewire), etc.)
3. Booted into safe mode minimal.
4. Ran WD in full scan, once finished I recorded the number of spyware detected but DID NOT remove any spyware and instead closed WD.
5. Ran Spybot S&D, recorded detected stuff, didn't remove any but closed.
6. Same procedure with Ad-Aware SE.
7. Same procedure with A-Squared.
8. Rescanned with WD this time QUARENTINING spyware; it failed to remove some GAIN stuff so I altered the number of spyware corresponding with WD.
9. Unquarentined spyware and scaned with Spybot S&D; removed all detected so its spyware rating stayed the same.
10. Restored the spyware detected using Spybot's Recovery Feature.
11. Scanned with Ad-Aware SE; all removed and then unquarentined.
12. Scanned with A-Squared all removed.
13. Rescan with WD removed fully what it could.
14. Rescan with Ad-Aware SE removed what it could.
15. Spybot S&D removed all.
16. For safe-guarding ran scans with TMAS and removed all detected afterwards, along with manually deleting what SpywareDoctor found.

Any more questions? Please reply back with them. This is excellent practice for my upcoming Discovery Channel Young Scientist Challenge and I would love to be vigorously interogatted and critiqued.