Optimizing DAT Performance: Smaller Is Better

By on Jan 06, 2015

This blog post was written by Vinoo Thomas.

I want to share some of the good work McAfee Labs did in the past year in optimizing and enhancing the V2 DATs (malware definition files, also known as AVV DATs) used in McAfee VirusScan Enterprise and other McAfee enterprise products. In 2014, we reduced DAT size by more than 45% to about 70MB, down from a high of about 132MB.

These enhancements have led to a big performance win: The reduction in DAT size automatically translates into faster system scan times and smaller DAT updates. Even more impressive is that these massive size reductions have been achieved while delivering consistently high protection effectiveness results in tests last year by AV-Test, AV-Comparatives, and NSS Labs.

V2 DAT Size 2014
Shrinking strategy
McAfee Labs evaluated its DAT signature categories and focused first on hash-based detections that had been added by our automation systems. Over time, human-authored generic signatures evolved to overlap most hash-based signature content, allowing for their safe removal without losing any detection capability.

The second strategy was to target signatures not seen in the field—mainly single-use malware deployed in common spam campaigns. The risk of seeing these old files in the field is very low. If these signatures were not seen via our McAfee Global Threat Intelligence (GTI) cloud telemetry, we moved them into the McAfee GTI cloud where they still provide protection but without the performance impact of constantly downloading unneeded data.

Antimalware engine releases such as the 5600 and 5700 engines used in McAfee VirusScan Enterprise and other McAfee enterprise products also allow us to port commonly used code in the DAT files to native engine code. Although in the past there were limitations to authoring generic detections on unsupported packers or file formats, new engines enable better decomposition of these formats, allowing researchers to create better generic signatures.

Continuing performance focus
The DAT optimization project was incredibly complex, requiring significant testing and validation to ensure DAT quality, safety, and consistently high protection effectiveness. Scan times are now back to pre-2011 levels without any product or technology uplifts.

As we continue to innovate, the ability to process V3 DATs—the successor to V2 DATs—will be integrated into all McAfee endpoint products. Today, V3 DATs are used by McAfee Endpoint Protection for SMB, McAfee Internet Security, and McAfee Antivirus Plus. V3 DATs further reduce DAT size. Currently, they are smaller than 30MB, providing even better system scan time performance while still delivering outstanding protection results!

V2 V3 DAT Size 2014

To learn more about the V2 DAT and the new V3 DAT, see KB82396: “FAQs for V3 DAT files.”

 

About the Author

McAfee Enterprise

McAfee offers industry-leading cybersecurity solutions for all business and enterprise needs. See our blog to stay up-to-date with the latest security trends

Read more posts from McAfee Enterprise

  1. Nice Article Vinoo, with useful information incorporated. Any plans of McAfee to utilize V3 DATs in Enterprise Products? As for the fact that they still do not use AMCore.

    • McAfee Endpoint Security 10 that debuted in Q4’2014 is the first enterprise product in the SMB space that incorporated the AMCore framework and V3 DATs. Future versions of VirusScan Enterprise will use the AMCore framework and V3 DATs going forward. VirusScan 8.8 and below versions will continue to use V2 DATs.

      -Vinoo Thomas

Subscribe to McAfee Securing Tomorrow Blogs