MITRE ATT&CK: Why Detections and Tainted Telemetry are Required for an Effective EDR Solution

December 13, 2018

| danbrown | Engineering & Tech

Following the MITRE ATT&CK™ Evaluation of endpoint detection and response (EDR) solutions, I've heard a lot of confusion surrounding the various terms MITRE used, particularly the terms "detections,” "telemetry” and the qualifier “tainted,” which MITRE applied to certain types of telemetry they observed in their evaluation. Each of these terms represents a very different type of data, and each plays a critical role in a successful EDR solution. Let’s take a look at each term to compare, contrast and understand the following:

Why telemetry alone is not the same as detection
Why detection and tainted telemetry are required for a truly effective EDR solution
How CrowdStrike delivers best-in-class results in all three categories

At the conclusion, we'll suggest possible improvements — taking concepts introduced in the ATT&CK evaluation and seeing how we might incorporate them into the ATT&CK framework.

We believe this could help reduce confusion around some of these concepts.

Telemetry: Comprehensive and Continuous Visibility is Table Stakes for EDR

A foundational capability of EDR solutions is to collect a rich set of raw endpoint telemetry that describes the processes and activities that have happened on an endpoint in great detail. MITRE defines telemetry as “minimally processed data that is accessible to an end user and directly indicates that the red team activity occurred after the user performs human analysis…” CrowdStrike Falcon® continuously collects comprehensive telemetry, even when malicious or suspicious activity is not being detected on the endpoint. This telemetry includes hundreds of different types of activity such as process creation, HTTP connections, service creation, logins and many other event types. As MITRE rightly points out, raw telemetry requires human analysis in order to identify (detect) malicious behavior. Telemetry is necessary but not sufficient to make an effective EDR solution.

Detection and correlation with telemetry are crucial to truly making analysts effective.

Detections: Intelligent EDR Turns Raw Telemetry into Actionable Insights

Detections (“General Behavior” and “Specific Behavior,” according to MITRE’s definitions) are automated techniques through which an EDR solution directly informs you, the user, of the existence of malicious behavior. Detections happen, for example, when a machine learning model fires on a malicious file, when a behavioral model flags a suspicious pattern, or when a human threat hunter identifies a stealthy chain of events. Despite what some test participants assert, raw telemetry, by itself, is not the same as detection. “Detection” requires analyzing telemetry, making a judgement that some activity is malicious, and raising an alert.

Conflating telemetry and detection is like missing the distinction between hay and needles. An EDR solution that focuses mainly on telemetry amounts to simply handing you the haystack and a magnifying glass. A complete EDR solution delivers a broad set of telemetry (to give the visibility necessary to support threat hunting and investigations), and highly accurate detections (to provide the insights necessary for incident response). CrowdStrike Falcon® delivered top-marks in both categories in MITRE’s evaluation.

Tainted Telemetry: Making EDR Usable with Understanding and Context

Finally, MITRE used a less familiar term, "tainted,” to describe a special class of telemetry.

Tainted telemetry is key to making analysts of all skill levels more effective. The term is used analogously in areas such as taint analysis, which allows researchers to find downstream effects assuming that attackers can control some range of memory addresses via data inputs.

MITRE defines the term this way:

“The capability detects the activity based on previously identified suspicious/malicious behavior that is related to or “tainted by the detection.”

In other words tainted telemetry eliminates a lot of hard work that would otherwise need to be carried out manually by a skilled human analyst. The CrowdStrike^® Falcon^® platform makes extensive use of this concept to streamline analysis. Falcon automatically surfaces for the analyst all existing telemetry that is connected to malicious detections. Tainted telemetry is presented automatically, without the need for additional logging, and without the analyst spending expensive cycles searching for it themselves.

In the example below from the MITRE’s evaluation (11.B.1), we see tainted telemetry in the form of a complete description of network connectivity from a PowerShell process that was previously identified as malicious by Falcon. Automatically serving up the right pieces of information, at the right moment in time, enables an EDR solution to be a powerful force multiplier. Tainted telemetry allows analysts to understand more threats, faster than they could with a less-capable EDR solution. This is critical when attempting to respond to threats within the “breakout time” window.

Fidelity and Utility in MITRE ATT&CK

I believe the distinction that MITRE usefully made through the use of the term "tainted" in the evaluation should be considered for inclusion into the ATT&CK framework itself.

A primary reason for the confusing discussions around MITRE’s evaluation is that the ATT&CK framework currently does not distinguish between two concepts I refer to as "fidelity" and "utility,” defined as follows:

Fidelity is how useful a piece of information is for distinguishing between malicious and benign activity.

A common way to think of fidelity is in terms of false positive rates. A "noisy" alert has low fidelity.
Utility, on the other hand, is how much value can be derived from the information assuming the activity has been confirmed as malicious.

You can think of MITRE ATT&CK techniques as providing some amount of fidelity and some amount of utility.

However, it is often the case that a technique is clearly useful primarily for one or the other. Take “Process Hollowing” (T1093) and “Standard Application Layer Protocol,” (T1071) for instance. These provide a high degree of fidelity and utility, respectively, as illustrated here: Process Hollowing is generally useful for finding malicious behavior. When we see it, it is a sign that bad things are afoot and further investigation is warranted. It may give us some useful information as well (utility), but once an analyst is alerted to malicious behavior via Process Hollowing, she will almost certainly wish to explore relevant adjacent telemetry for additional information. Standard Application Layer Protocol, on the other hand, is a technique (if we can call it that) which consists of things like network HTTP connections. Nobody would want to be cursed with an alert for each and every HTTP connection in their security environment. An ideal solution would automatically correlate for the analyst HTTP connections that are associated with the detected malicious behavior — and that is exactly what the concept of tainted telemetry does. It presents relevant telemetry to the analyst as context for detections without the need to query for it, providing utility. To summarize, I suggest that the community consider adding a distinction like this to the ATT&CK framework. I believe it would help improve the general understanding of the role of techniques employed by our adversaries. I also think that showing the distinction between techniques that should be detected, as opposed to activity that should be collected as telemetry (tainted or otherwise) would have the added benefit of discouraging potentially harmful or misleading uses of the framework such as viewing it as a "coverage" map for detection or measurement of security. Kudos to MITRE for including a useful concept like "tainted telemetry" in the ATT&CK Evaluations, and perhaps we can carry that useful notion back into the ATT&CK model itself.