After having some viruses get through 2 different machines using AVG plus other anti-spyware and firewall programs, all set up as recommended on this site, I started to have major doubts about AVG's effectiveness, and also wonder if someone was doing comprehensive comparison tests on these different programs to see how well they really perform.

There seems to be a fairly strong preference by the more experienced people in this forum for AVG, but I suddenly have major doubts.

I just searched online yesterday and came across (so far) this one comparison test on PC World :

http://www.pcworld.com/article/id,124475-page,1/article.html

How much faith we should put in PC World's testing I'm not sure, but comparing 10 products, 3 of which were free, AVG finished last (10th) in their test, with a "Fair" rating of its performance. AntiVir finished highest of the free products (7th overall) in performance (Good). BitDefender was highest overall, McAfee 2nd, then Kapersky. Norton was 5th. Trend Micro was the worst performer of the paid products, finishing 9th overall, performance about the same as AVG's.

Here are some of the results breakdowns: (I know, it's not nearly enough detail)

1. Bit Defender

Performance: Superior
WildList viruses - 100%
AV-Test zoo threats - 95%
Heuristic detection with one-month-old signatures - 56%
Heuristic detection with two-month-old signatures - 38%
Scan speed (in seconds) - 556
Outbreak response time - 2 to 4 hours

7. AntiVir

Performance: Good
WildList viruses - 100%
AV-Test zoo threats - 95%
Heuristic detection with one-month-old signatures - 11%
Heuristic detection with two-month-old signatures - 6%
Scan speed (in seconds) - 265
Outbreak response time - 4 to 6 hours

8. AVG

Performance: Fair
WildList viruses - 100%
AV-Test zoo threats - 80%
Heuristic detection with one-month-old signatures - 8%
Heuristic detection with two-month-old signatures - 4%
Scan speed (in seconds) - 354
Outbreak response time - 8 to 10 hours

I don't know what WildList viruses are, or zoo threats, or signatures (are those updates?). And I'd be very interested to hear details like how many total viruses were tested, and if they're fairly representative of what might happen in the real world.

AVG is usually recommended around here, but I'm wondering how people here come to that conclusion.

---------------------

I just came across these 2 sites that seem to be much more serious than PC Worlds tests:

www.av-comparatives.org and www.checkvir.com

AVG does better on av-comparatives tests than in the PC World test, but not very well against polymorphic viruses, which I think are the type we got hit with (W32 type viruses). On those types, theyre near the bottom of the list.

AVG was a good AV program a few years ago when they didn't have a paid version. As free AV programs go, AntiVir is better now but AVG is still living off their old reputation. That's not unusual. You see it with spyware scanners too. SpyBot & Ad Aware were the best a few years ago and some still swear by them based on that. AVG AntiSpyware & Super AntiSpyware are clearly better now but SpyBot & AdAware will continue to live off their old rep. The answer? Do your own research and make your own decisions. Also, online virus scanners are a good second defense. I use Panda Nano Scan and I'm going to try Eset shortly.

I think you have begun to answer your own question:"And I'd be very interested to hear details like how many total viruses were tested, and if they're fairly representative of what might happen in the real world."
That there are many tests and that each arrives at different results, or ratings, indicates that each testing group uses different sets of virus and other criteria for testing. The better groups will provide detailed information about how the tests were conducted, the procedures used, and the data against which each application was run.
Cheers,
John

I notice that the tests run by AV Comparatives and Checkvir are all about performance in catching, protecting against or eradicating viruses, while the PC World comparison includes this as well as things that aren't as clear cut, such as 'ease of use' or design, technical support, price, etc. While these might be very important to many customers, for me what's far and away most imporant in an anti-virus program is how well it prevents viruses from getting on my machine. If it's protection is about the same as a competing product but the other one is easier to use, sure then I'd choose the easier one.

Anyway, I think interface is something anyone can check out themselves and make their own decision. But an individual user can't try a program out against 500,000 different kinds of viruses.

The more I study the tests, the more I see that AVG doesn't do a great job of detecting viruses that AVG hasn't already identified -- in other words, it doesn't catch viruses that would be picked up by their warning signs. AntiVir and most others do a much better job in that particular department, as well as do about as good a job in the areas AVG also does well. This seems to be a major weakness of AVG.

I've seen some people say that the tests all have different results... Well of course they do, but the trends also stand out. One is what I just mentoined about AVG's heuristics, which seems to show up in all the serious tests. And the ones that tend to be among the best in one test also tend to be among the best in others. Maybe not the exact same order, but in general they're in the same ballpark. It doesn't mean the tests are completely irrelevant, no more than baseball standings would be irrelevant because the Yankees are on a hot streak now while in May they were losing.

Here's another factor when rating AV testing: System Resources Used

Having a great AV won't mean much while your system is bogged down trying to run it in the background.

I've found it increasingly difficult to extract useful information from tests, reviews, and user reports for a number of reasons, one of which arises from the blurred boundary between the things that an antivirus program might be expected to detect, and the things that an antispyware program might be expected to detect.

For example, I look at this vague combination of 'AVG plus other antispyware programs', and I wonder how vulnerable that might be, simply because it's not an integrated combination. For instance, I personally would not feel secure using AVG Free plus (let's say) Spybot and AdAware to scoop out the spyware. Not because they aren't all good programs in their own ways, but because they were never designed to operate together as a unit. They come from a time when a virus was a virus and spyware was spyware (and when the spyware was possibly more like an irritant than a threat). My impression is that things have changed a lot since then, and what is needed is some kind of integrated approach (I adopted AVG's AntiMalware myself, but it's the basic principle that counts) which combines real-time protection against the whole range of threats (plus, of course, other antispyware scanners as a back up resource.) We could be using the 'best' antivirus and the 'best' antispyware, and still leave gaping holes in our defences because of that uncertain area in the middle ground; we simply don't know. At least, I don't know.

As far as reviews are concerned - what are we to make of reviews (apparently sometimes using spyware simulators) that highlight what are seen as serious limitations in a program like SUPER Antispyware, when so many people, dealing with real threats in the field, report time after time that the program is extremely successful at cleaning out quite heavily infected systems in practice?

At the moment there seem to be too many variables to get a clear picture.

My intitial reaction on reading your idea about the blurred line between viruses and spyware, and how there are likely to be gaps between what AV and ASW programs cover you against, was that you were exactly correct. But after a day of thinking about it, it struck me that the blurred lines probably lead to AV programs trying to cover more of the spyware that acts like a virus, and anti-spyware likewise being more concerned about anything in the gray area that might be considered by some to be part of their territory. So if you had the best AV and best ASW, you're probably going to end up with two programs trying to protect against some of the same threats--an overlap, in other words. The idea behind having a number of different anti-spyware programs is to get blind spot coverage for what the other doesn't provide, so if you went with one all-in-one package, I would think it's more likely to have blind spots than if you used a single AV program together with 2 or 3 ASW programs.

I agree that ideally, there shouldn't be different programs only focused on one type of threat or the other, but rather have one program that just considers protecting against threats to be its job regardless of how they're categorized. I'm not sure why things went in that direction in the first place.

It actually might be an interesting experiment, to test whether an all-in-one package like you describe would perform better than 2 separate ones working together o the same machine. My gut says that you'd probably get better protection with the 2 specialized programs than the all-in-one program.

What I think is good about the comparison tests is that they do tend to point out certain weak spots in the different programs. One's scan running time is longer than another's. Some are better than others in identifying threats that aren't already known by the software, or contained in updates. And so on. What I'm finding is that there do seem to be some organizations out there doing strict and sensible testing.

That's a reasonable guess, but my point is that we don't actually know, because we don't know what criteria were used in making the decisions by the two separate programs.


There are two distinct issues, I think:

1. Real-time protection. This is where I'm most worried about 'the grey gap', because I suspect most of us are likely to have only one AV and one AS running in real time in order to avoid excessive use of resources and to minimise the potential for conflict between programs. Now, let's suppose you have AVG Free as your AV, and you're looking for a single real-time active AS program (which you'll probably have to pay for). You're only going to choose one - but which? My argument would make AVG AS the natural choice, because together with AVG Free it makes an integrated solution. If you choose Windows Defender, for example (which is free), you have all the uncertainty of the grey gap in your RTP. And I would far, far prefer to intercept malware on the way in, than to find it afterwards with an on-demand scanner, and be faced with the problem of removing it.

2. On-demand scanning. This isn't the same problem because you can use as many separate scanners as you want, and this is where I completely agree with your argument. With a whole battery of them, you're very likely to cover all the gaps, and I have half a dozen scanners that I use regularly. But this is for scanning only (and removal if necessary). It's not so simple for real-time active protection.


In the case of the AVG solution (which is what I'm presently using, in the form of the combined Internet Security suite), I think you can kill two birds with one stone. AVG AS is based on Ewido - which was a generally highly regarded specialised antispyware program. When AVG combined that with their antivirus program to make AVG Antimalware, I think they were having a shot at getting the best of both worlds. The user knows he's using software based on two reputable programs (maybe not the very best, but still pretty good), but also that 'grey gap' won't exist. Well - it might exist in error (that will always be possible); but there is the advantage that the AV and AS sections are no longer developed separately, but together.

All we can do is read the tea leaves as well as we can and make our own decisions in the end. There is no 'best' solution, because no matter how many reviews we read, we simply don't know enough. This time next year I may find myself making a different set of decisions.

i wonder why PC world never mentioned about Nod32 ?? i am using it now but somehow a worm call W32/Sohana-R ( SSVICHOSST.exe ) got undetected into my pc... maybe the guys at at nod32 company is slacking ....

Well, I think ideally this approach would be nice. The main problem I see with it is that AVG, according to the comparison tests I've viewed so far (and I'm not talking about PC World but on the real hardcore testing sites like AV-Comparatives, Checkvir, WestCoastLabs, AV-test.org, etc.), AVG seems to be very weak in the category of protecting against unknown viruses -- in fact it's one of the worst performers of all the top AV programs out there in this category.

I think it's natural for people to make judgments according to their own personal experience, rather than look at a more comprehensive collection of evidence like serious testing. To me it seems like most people posting here rely on personal experience - which is not a scientific approach to the problem. And actually I have to confess that I'm even doing that myself since within a week I discovered 2 machines running AVG infected with 2 different viruses.

But if a lot of people who've been lucky not getting infected with AVG post here claiming it's proven to be better, and ignore any kind of serious testing that's being conducted on these products (or try to dismiss them for whatever reason, possibly because it doesn't happen to jive with their prejudice on the topic), what you're going to end up with is the creation of a myth. A lot of people wanting AVG to be 'the best' without any good reason for saying so, all reinforcing each other.

I scan these forums and see AVG recommended again and again. Then I look at the testing sites and see it not getting the highest marks, not close, and I look at these 2 computers that went down with viruses while AVG was supposedly protecting them. It also makes me wonder about a lot of the other advice being given here.

To me it just seems like a no-brainer that the quality of the AV programs should be based on how well they perform in a real-world-simulating test environment. Nobody here can conduct exhaustive, comprehensive tests by themselves -- it's too big a job. Any criticisms of that approach, doing serious testing, should be toward improving the quality of the testing, and not tyring to claim it's futile. To try to argue that testing is pointless, it really seems like a head-in-the-sand attitude.

Anti-virus programs like AVG and avast are popular choices because they are free. While they may not be the best when compared to "paid for" products, they certainly are choices to consider when looking for free alternatives.

Keep in mind that no single product is 100% foolproof and can detect and remove all threats at any given time. The security community is in a constant state of change as new malware infections appear. Each vendor has its own definition of what constitutes spyware and scanning your computer using different criteria will yield different results. The fact that each program has its own definition files means that some malware may be picked up by one that could be missed by another. Thus, a multi-layered defense using several products (including an effective firewall) to supplement your anti-virus provides the most complete protection.

I'm not sure whether you're making a general statement here, or responding specifically to my comments. If the latter - well, those are not my opinions, as you state them there. I don't believe testing is pointless; I just think it's imperfect, and the results need reading intelligently and with due circumspection. To look at a list of scores in a review and say 'this is the best' and 'this is the worst' may be seriously misleading. At the very least, you have to read the small print. To be scientific, as you suggest, you have to be in a position to control the variables, and we're still some way off being able to do that.

And as in most walks of life, personal experience must count for something. When you've actually seen a heavily infected computer completely repaired by Superantispyware (when Defender was unable to discover any problem at all, and Spybot failed to remove what it found); when a highly knowledgeable friend reports a very similar experience with the same program, on two separate occasions on two different computers; - when that happens, it does have a significant effect on your confidence in the program. And it would also raise important questions about an unfavourable review of it. Don't you think?

Incidentally, those not very impressive test performances of AVG that you mention do bother me, as an AVG user. I shall certainly take them into account when I decide what to do when my present subscription runs out. But they are not the whole story, and my AVG Security suite is not my only line of defence. Quietman7's point about the layered defences is crucial, I think, regardless of the various choices we each make about the individual layers.

No sorry I was speaking generally, that it strikes me as the way many here approach their choices and recommendations. I think 'scientific' may not be the right word -- it is more a case of interpreting the trends in the different tests, and giving some leeway either way, since the program may have overperformed or underperformed in that particular test because of the luck of which viruses were thrown at it. Though if some program is consistently getting very low scores across tests run by different organizations, and over the course of time, I think that has to be taken seriously. As I scanned AV-Comparatives and CheckVir last night, the one most obvious standout program seemed to be NOD32, which seems to always be near the top in most categories. Other programs fluctuate more, with some excellent results on some tests and poor ones on others (Avira AntiVir is a good example of that).

I don't think anyone can discount their own personal experience, but you have to make an attempt to balance it with some wider view of the situation -- I mean obviously one person can't get a broad experience of viruses on their own. What I should have said was that it seems like most people here rely SOLELY on personal experience, ignoring tests.

I wasn't aware of those bad test results for SuperAntispyware. I'd be curious to check that out, have only found AV tests so far.

Since reading posts here more in the past week, I actually have downloaded and installed that program.

A lot also depends on the nature of the personal experience. For example, Mr X's statement that 'I have AVG and I've never been infected' is of little value as evidence in favour of AVG (he might actually have infections that haven't been detected; or he might have extremely safe internet habits and has simply avoided exposure to much). However, in the Superantispyware examples I mentioned, here were three instances of infected computers completely cleaned by a single program, where other programs had failed, either wholly or partly. That's powerful evidence in favour of the program.


Here's a link to the official Superantispyware response to it, which also provides a link to the source of the review in question:
http://forums.superantispyware.com/viewtopic.php?t=779


Just my opinion of course - but I think you did yourself a big favour in doing that.