Mac anti-virus testing 2013 – The Secure Mac

There are nearly as many anti-virus packages for the Mac as there are households of malware, and a continuing query amongst Mac customers is whether or not to make use of one and, in that case, which one to make use of. Final November I started a challenge to check Mac anti-virus packages to see what malware they’re able to detecting. This doc describes the second spherical of testing, wherein I have a look at a complete of 20 completely different anti-virus packages utilizing considerably completely different strategies than these utilized in the primary check.

It is crucial, earlier than beginning with dialogue of the check, to level out the relevance of this check. This isn’t an try to match anti-virus packages throughout the board. This check examines solely a specific side of the anti-virus engines being examined: what malware is detected by a handbook scan. This check didn’t try to check how nicely any engine blocks an energetic try at an infection. It additionally accommodates completely no details about the function units, efficiency and stability of any of the examined engines. Don’t try to make use of this check as the only metric of evaluating anti-virus software program. Take into account that I might actively advocate in opposition to a number of of the anti-virus packages that scored extremely on this check!

Strategies

On this check, a complete of 128 samples have been collected, containing gadgets from 24 completely different malware households. Samples have been organized into folders primarily based on malware household. Samples that got here from VirusTotal had names consisting of the SHA1 “fingerprint” of the file. Samples that didn’t initially come from VirusTotal have been uploaded to VirusTotal, then given names similar to the SHA1 title assigned by VirusTotal. This was achieved to simplify identification of which malware was detected. Any samples that consisted of archives (zip information, disk picture information, and so on) have been expanded/opened, and each the archive and the contents have been positioned in a sub-folder consisting of the SHA1 “fingerprint” of the archive.

Makes an attempt have been made to make sure that all samples have been legitimate samples. Generally, VirusTotal outcomes aren’t conclusive, and samples will probably be recognized as malware that actually aren’t. Numerous samples have been rejected from inclusion within the testing through the assortment part. Two gadgets (parts of the DiabloMiner app) have been eliminated after testing, when it was proven that no anti-virus software program detected them, and after figuring out that DiabloMiner is definitely a legit program misused by DevilRobber.

Testing was achieved in a digital machine in Parallels. A base Mac OS X 10.8.2 system was arrange in a digital machine, absolutely up to date and with no third-party software program put in. A snapshot was created of this method. Then, over the course of a number of days, 20 completely different anti-virus packages have been obtained and put in in recent copies of this digital machine, ending with 20 completely different snapshots in Parallels, every containing this base system and one of many anti-virus packages to be examined. As soon as set up was full, a single day was chosen to open every snapshot and replace every anti-virus program, then save a brand new snapshot of the up to date state. The ultimate end result was a set of similar methods, every with a totally up-to-date copy of one of many anti-virus packages as of that individual date.

As soon as that was achieved, by shutting off community entry, testing may proceed over a number of days with out altering the outcomes. Every system was run in Parallels, and the folder containing the malware was copied onto the desktop of the check system. (If essential, any energetic or on-access scanning was disabled to permit this to be achieved unimpeded.) Then, a handbook scan of that malware folder was achieved. Most anti-virus software program allowed the collection of a selected folder for handbook scanning, however some required scanning all the person folder and even all the digital onerous drive. In any case, the one malware on the system was within the malware folder, so the outcomes have been equal.

After testing, the outcomes have been tabulated. This was a troublesome course of in some circumstances, as many anti-virus packages present no choices for saving scan outcomes. (Some present command-line instruments that can be utilized for scanning, however solely the GUI scanner was used. That’s what the common person can be utilizing, and utilizing some command-line instruments might result in claims of variations in scanning between the command-line and GUI variations.) Within the case of malware samples consisting of a number of information, the malware was thought-about to have been detected if any single merchandise within the folder containing the pattern’s information was recognized.

Knowledge

The whole information could be downloaded as both a Numbers spreadsheet or a PDF file. (An Excel file was not offered as a result of among the conditional formatting guidelines that make the information extra readable weren’t included.) Detection charges (outlined as the proportion of samples that have been detected) assorted extensively, from 98% down to six%. Half of all anti-virus engines examined carried out at 93% or higher, and nearly 3/4 of the engines obtained a “passing grade” (79% and up). Six carried out at 66% or decrease.

There have been 59 samples of what can be thought-about energetic malware, omitting malware that’s “extinct.” Of these samples solely, the detection charges assorted the total gamut, from 100% right down to 0%. The identical ten engines within the high of the testing when together with all samples as soon as once more carried out at 93% or higher with energetic malware, and a full 3/4 of the engines carried out at 73% or higher. Solely 5 fell under 60%, with one holding the file by not detecting any energetic malware in any respect. For probably the most half, the proportion of whole malware detected was very near the proportion of energetic malware detected for every engine, though variations as excessive as 15% have been seen.

Among the many samples of malware, detection charges assorted from being detected by all 20 engines right down to solely being detected by 7 engines. On common, samples have been detected by about 15 engines.

Conclusions

Though it is very important take into account that this is just one measure of the standard of every of the examined anti-virus engines, it isn’t an unimportant one. Clearly, though it isn’t possible for any anti-virus software program to detect 100% of all malware, a very good engine ought to be able to coming as near that quantity as potential. That is very true within the Mac world, the place the restricted variety of malware households implies that detection charges of very near 100% ought to be potential. As anticipated, some engines did certainly carry out to that normal.

Different engines didn’t fare so nicely. Nevertheless, it is very important take into account that Mac OS X already does an admirable job of defending in opposition to malware. At the moment, there isn’t any identified malware able to infecting a Mac working a properly-updated model of Mac OS X 10.6 or later, with all safety settings left on the default (at a minimal). The position of anti-virus software program should be considered, and a few compromises in detection price could also be fascinating to get desired conduct (or keep away from dangerous conduct). Somebody who desires a low-impact engine for scanning e-mail messages for Home windows viruses can have very completely different wants than somebody who wants to guard a pc from an irresponsible teenager who will obtain and set up something that catches his/her consideration.

When selecting anti-virus software program, all the time take the total set of options under consideration, in addition to in search of out group suggestions relating to stability and efficiency. Ensure that you know the way to uninstall the software program earlier than putting in it, in case it causes issues and must be eliminated.

For extra on the subject of defending your Mac in opposition to malware, see my Mac Malware Information.

Notes

Why change the strategies?

In my first spherical of testing, 51 samples have been examined in opposition to 16 engines. That pattern measurement was actually too small, although it’s troublesome to seek out a lot of samples of Mac malware, since there are so few malware households for the Mac. There have been additionally a number of different issues with that pattern set, together with one Home windows .exe file that was mistakenly recognized as Mac malware and included erroneously (although it ought to nonetheless have been detected as Home windows malware) and some minor disagreements about whether or not gadgets ought to or shouldn’t be included.

One major aim of my second spherical of testing was to not solely scan a bigger set of samples, however to extra fastidiously display every pattern to make sure that it was acceptable for inclusion. Though there’ll nonetheless in all probability be some dialogue of whether or not sure gadgets are acceptable or not, this set is general a lot higher-quality than the earlier one.

One other downside some folks had with the unique check was that some samples have been archives of various sorts (principally zip information). Not all anti-virus engines are able to trying inside archives, and of these which are succesful, not all will accomplish that by default. Because of this, I selected to develop any such archives and embody each the archive and the contents within the pattern set.

One of many greatest points needed to do with the best way the testing was achieved. I initially did all of the testing in a one-day interval, whereas my pc was booted right into a check system on an exterior onerous drive. This meant that the testing surroundings ended up being destroyed when the testing was accomplished. That meant that there was no strategy to settle problems with what engine had been used or provide different unrecorded data. Within the second spherical of testing, I modified how I carried out the exams to forestall this concern. I selected to make use of a collection of snapshots in a Parallels digital machine. This meant that, by slicing off community entry and opening a selected snapshot, I may repeat testing below the identical situations and collect extra data that is perhaps requested sooner or later.

Anti-virus software program examined

The next anti-virus packages have been examined:

Objections

There are a number of objections that some might have with this check, so permit me to handle them upfront.

First, some will object that it is a somewhat synthetic check, and never a real-world one. Though it could clearly be higher to check by making an attempt to contaminate a system with a wide range of malware and figuring out whether or not every anti-virus software program would block the an infection, that is impractical. Not solely wouldn’t it be exceedingly time consuming with only some samples, however it could be pretty meaningless as nicely, since Mac OS X is presently capable of block all identified malware by means of a wide range of strategies. Testing with static samples could also be much less informative, nevertheless it does give priceless details about the completeness of every engine’s virus definitions database.

The pattern measurement may additionally be insufficient for affordable testing. 128 samples is much better than the 51 samples from my earlier check, nevertheless it’s nonetheless a bit low. After all, so is the variety of malware households for the Mac. By my rely, there are solely 35 completely different malware households which have ever been able to affecting Mac OS X, and given such shortage of malware households, it’s to be anticipated for samples to be onerous to come back by for somebody not affiliated with any anti-virus firm. My opinion is that the samples used are a reasonably good collection of malware, however in fact, it may very well be improved on sooner or later.

Lastly, some might object to the truth that greater than half of the samples are what can be thought-about “extinct” malware, since such samples are now not an actual menace to anybody. Nevertheless, details about what malware has been detected traditionally by an anti-virus engine is vital for predicting future accuracy. In truth, trying on the information, it’s clear that there’s a correlation between general detection price and detection price for energetic malware solely. There’s additionally the truth that some folks could also be in search of anti-virus software program for outdated, legacy methods that will have malware infections from years previous nonetheless in place. After all, separating out the energetic malware solely does have its makes use of, akin to figuring out which packages are enhancing and that are falling behind, which is why I included a abstract of these numbers within the information in addition to the general statistics.

Particular circumstances

There have been a number of particular circumstances in numerous features of the testing.

iAntivirus apparently doesn’t function any form of mechanism for updating its definitions. (That is confirmed by a Symantec worker within the Norton boards.) Which means that, though I used to be utilizing the newest model of iAntivirus, its definitions have been greater than two months outdated. (Which might clarify why it did a lot worse in opposition to latest malware!)

The MacKeeper trial model refused to replace the virus definitions except it had truly been registered. Fortuitously, I had been given a serial quantity by Zeobit not too long ago, so I went forward and registered in order that I may replace the definitions. This was the one business product that was not utilized in its time-limited trial mode.

F-Safe evidently has a little bit of an issue with its GUI when working in Parallels. It frankly doesn’t work in any respect. Fortuitously, F-Safe tech help was capable of give me a work-around that allowed me to check it anyway, by enabling display sharing within the digital machine after which connecting from the “actual” system on my Mac and controlling the software program from there. A bit bizarre, nevertheless it labored. Observe that that is particular to working F-Safe in Parallels, and isn’t a difficulty when put in conventionally.

Updates

There have been a pair minor transcription errors (malware that was marked as not detected when it truly was) that have been delivered to my consideration and have now been mounted within the information. I will probably be reviewing the information additional to ensure there aren’t another errors. Though such issues are sure to occur when combing by means of 1000’s of information factors, which needed to be collected by means of screenshots in lots of circumstances, my apologies to everybody for the error!