Today sees the official formation of the Anti-Malware Testing Standards Organization (AMTSO) which has come about following an industry wide concern about the lack of any real-world standards that apply to anti-malware solutions when it comes to testing. Why is this important? Because unless the testing methodologies used to evaluate anti-malware are doing an effective, and consistently so, job then the product reviews that end up in magazines and published on the web are going to be incomplete, inaccurate and sometimes simply misleading. This has become an increasing pressing problem to address as anti-malware solutions become ever increasingly complex themselves in order to best secure systems against the maturing malware threat.

AMTSO itself is purely focused on addressing this global need for an improvement in the objectivity, quality and relevance of these testing methodologies, and as such is looking to promulgate universally adopted standards and guidelines. AMTSO promises to:

  • Provide a forum for discussions related to the testing of anti-malware and related products
  • Develop and publicize objective standards and best practices for testing of anti-malware and related products
  • Promote education and awareness of issues related to the testing of anti-malware and related products
  • Provide tools and resources to aid standards-based testing methodologies
  • Provide analysis and review of current and future testing of anti-malware and related products

As Andreas Marx from the highly respected site says "well executed and comprehensive tests will light the way to better products -- it is not only the developers who contribute towards the improvement of products. Most developers focus on the aspects of a product which are used to compare and rank products and to finally perform better in such kind of tests. Thus, it is essential for testers to move on the next level of product testing, focusing on everything besides the "traditional" signature detection. If this doesn't happen, an entire industry might run into trouble and with it, billions of users may be misled by inadequate tests."

About the Author

As Editorial Director and Managing Analyst with IT Security Thing I am putting more than two decades of consulting experience into providing opinionated insight regarding the security threat landscape for IT security professionals. As an Editorial Fellow with Dennis Publishing, I bring more than two decades of writing experience across the technology industry into publications such as Alphr, IT Pro and (in good old fashioned print) PC Pro. I also write for SC Magazine UK and Infosecurity, as well as The Times and Sunday Times newspapers. Along the way I have been honoured with a Technology Journalist of the Year award, and three Information Security Journalist of the Year awards. Most humbling, though, was the Enigma Award for 'lifetime contribution to IT security journalism' bestowed on me in 2011.

So an industry group consisting of a few companies selling malware detection software is claiming that most such software (read, software produced by their competitors) doesn't work properly?

Sounds like advertising to me, launch a new "quality logo" or "certification" to show what programs are "guaranteed good" and apply it to your own products only.
A lot of potential customers (and given some money magazine and website editors too) will happily fall for it, either not knowing better or in the case of those editors looking the other way after being paid to do so.

Nothing new there, been going on in many areas for a LONG time. Just don't trust "independent quality certification" on face value.