MITRE APT29 Evaluation – Importance of Prevention in Endpoint Security

By and on Apr 28, 2020

In our recent Racing with Cozy Bear blog, we covered the concept of Time Based Security and highlighted the value protection brings to the defender. This is not to say that blocking an attack removes the threat actor from the equation. Attack-blocking protection slows down the offender, buying the defender valuable time to respond. There are three reasons for this:

  1. Blocking an advance, forces the offender to change their approach and try again
  2. Block-level detections are inherently high fidelity, elevating their priority for defenders
  3. Defenders can focus on other higher priority detected events that have not been blocked

As part of the APT29 evaluation, MITRE did not allow vendors to deploy products in blocking mode as not to interfere with the test. However, they did allow for the deployment of such technologies in non-blocking mode and for participants to highlight scenarios where products would have blocked.

Block-level detections bolstered McAfee’s performance more than any other vendor.

In future evaluations MITRE has stated that protection results will receive their own categories, but during the APT29 evaluation, MITRE captured block-level detections as footnotes as shown in Figure 1.

Figure 1 – Example of Block footnote

From a defender’s perspective, detections that are more definitive are more actionable with increasing value. In keeping with Time Based Security, Host Interrogation has been brought into the following chart; a visual representation of detection types from the evaluation.

Figure 2 – Time-base representation of the value for each detection type

The scope of MITRE’s APR29 evaluation covered 20 major steps across all participating vendors, covering 57 techniques spread across 134 sub-steps. One major step was removed due to emulation challenges, leaving 19 major steps.

The following chart shows plots the highest-ranking detection from each participant. Each step represents the major attacker milestones as emulated, and an opportunity for the defender to protect, detect, and respond.

Figure 3 – Time-Base Security view of best coverage per major step

Another representation of this data is to aggregate these top-detection values for each participant. Here a block modifier is applied to fully represent non-blocking detections as well.

Table 1 – Block Modifier value assignments
Figure 4 – Aggregate Time-Base Security view

Not only did block-level detections bolster McAfee’s performance more than any other vendor, but MVISION Endpoint was the only solution to report such detections on several attack steps.

An example of this in action was captured during:

  • Step 11 – Initial Compromise
    • Technique T1140 (Deobfuscate/Decode Files or Information)
      • Sub-step 11.A.10 (Decoded an embedded DLL payload to disk using certutil.exe)

Living off the land binaries (aka lolbins) are native operating system files that can be (ab)used for more than their original intent. Adversaries are known to use them to bypass security controls since most of these programs are otherwise trusted. Either used in a macro or from the command-line, there are several examples available. A popular choice by groups such as APT28, Turla, Oilrig, and APT10 is the ‘certutil.exe’ tool. Originally intended to query for certificate information or configure Certificate Services, it can also be used to obfuscate/de-obfuscate data (T1140) or remote file copy (T1105) to download files.

At the time of this writing, MITRE has 70 report references for T1140, indeed making it a go-to technique for many offenders. Figures 5 and 6 were captured during the evaluation of this technique.

Figure 5 – JTI rule prevents live off the land attacks using certutil.exe during sub-step 11.A.10
Figure 6 – JTI rule prevents live off the land attacks using certutil.exe during sub-step 11.A.10


While this coverage was provided by MVISION Endpoint, the underlying technology involved is the same in McAfee Endpoint Security 10.7.

Ultimately, coverage is about time. MITRE’s APT29 evaluation in its own way highlighted McAfee’s Time Based Security protection and McAfee’s distinction in block-level detection. Buying time by throwing a speed bump into the path of a speeding Cozy Bear can be the difference in winning the race for security.

*All data is from:


© 2018 – 2020 The MITRE Corporation. This work is reproduced and distributed with the permission of The MITRE Corporation.

About the Author

Craig Schmugar

Craig Schmugar is a Sr. Principal Engineer at McAfee. Since joining McAfee in 2000 he has worked in different areas of research, from Malware Operations to Innovation Research. More recently Craig has been focused on endpoint product efficacy; assessing detection effectiveness and seeking opportunities to make it stronger. He has over a dozen pending or ...

Read more posts from Craig Schmugar

Christiaan Beek

Christiaan Beek is the Lead Scientist & Sr. Principal Engineer of the Enterprise Office of the CTO. He is leading the strategic threat intelligence research with a focus on inventing new technology, research techniques and models. Visionary and serving leadership is at the core of his day-to-day job, getting the best out of people and ...

Read more posts from Christiaan Beek

Categories: Endpoint Security

Subscribe to McAfee Securing Tomorrow Blogs