ATT&CK EVALUATIONS:

Using Evaluations

How To Use & Extract Value from MITRE Engenuity ATT&CK® Evaluations

Knowing how to review ATT&CK Evaluations best empowers you to pick the vendor that is most relevant to your organization’s needs and helps you understand your tool. Many people stop their analyses of ATT&CK Evaluations at the summary, but it’s important to go beyond because:

  • Each section informs the other for a meaningful analysis.
  • The sum of what you discover in the details is greater than the whole found in single-number results.

Remember, we don’t compare or rate providers or tools. We only evaluate individual tools.

Drone flying under sunset sky
ctid collaborative r&d

Evaluation Complete

ATT&CK Evaluations Emulated the Turla Threat Group

ATT&CK® Evaluations has completed emulating and evaluating enterprise providers for defenses against the Russian-based threat group, Turla, that has infected victims in over 50 countries. Evaluation results are now available.

Tips To Best Understand & Leverage ATT&CK Evaluations

ATT&CK Evaluations are a starting point.

We use an open-book and minimally sized environment to understand baseline capabilities of solutions. Operationalization of these solutions is important to consider in the context of your organization, including false positive generation.

There are no winners.

The goal of ATT&CK Evaluations is to show the different capabilities of each provider.

Not all techniques are created equal.

A technique detection for credential dumping may not have the same value as a technique for process discovery due to the severity of the actions. The category gives you a general idea, but you should dive into the details to understand the technique and detection.

Look at the UI/UX view for more than aesthetics.

Consider how the UI/UX would work, not just the look and feel of it. Are you seeing the information you’d need in real-time?

Determine how a provider’s tool presents itself to your analysts.

Thumb through provider screenshots to see if you’re being presented the data you need and review a tool’s overall usability. Is it easy to find the data you want to see in real-time?

 

See if a tool detects and prioritizes known threats to your organization.

Understanding a tool’s detection and prioritization capabilities will help you know if you’ll get an alert at the right time and in a way that allows you to trigger effective deterrence.

 

Understand the data sources: Are they effective?

For example, from what data source was a detection triggered, and how does that trigger fit into your cybersecurity needs and tactics?

 

  • Evaluation Overview Page

    Each Enterprise evaluation has available information specific to those evaluations. The overview page for each round provides high-level information and includes:

    • Links to the results
    • A list of participants
    • A description of the adversary
    • Emulation notes
    • The technique scope
    • Environment notes
    • Detection categories
    • Related resources about the evaluation
    Access an Enterprise Evaluation Overview
  • Detection Summary Screenshots

    A detection summary shows a collection of screenshots from the provider. This allows users to quickly review the UX/UI of the tool.

  • Detailed Results & Screenshots

    Within Evaluations results for every provider, there are specifics on each scenario and details for every step and sub-step of the emulation plan including:

    • Tactic, technique, and sub-technique information
    • Detection categories
    • Detection criteria and data sources
    • Screen shots of detections within the provider’s environment
  • Protection Results

    These are linear tests with a defined start and end. They are opt-in and not every result includes them.

  • Participant Configuration

    This includes provider product versions and descriptions used in the evaluation.

  • JSON (Java Script Object Notation) File

    The JSON file can be downloaded to dive more deeply into results and build internal analytics to further parse data.

Call For Participation Now Open

ATT&CK EVALUATIONS WILL EMULATE Ransomware with an Introduction to macOS

Enterprise 2024 will focus on ransomware with an introduction into macOS and targeting by the Democratic People's Republic of Korea (DPRK). This year, we examine common behaviors that are prevalent across prolific ransomware campaigns, such as the abuse of legitimate tools and efforts to evade defenses.

Call for Participation is open through April 30, 2024.

mitre engenuity attack evaluations enterprise 2024.

Evaluations by Industry

The ATT&CK Evaluations program continues to develop new methodologies, and open new rounds of evaluations. Currently, there are four types of ongoing ATT&CK Evaluations available:

Enterprise

ATT&CK® Evaluation for Enterprise empowers end-users to make more informed decisions on endpoint detection capabilities by articulating how each vendor can protect against or detect adversary behavior.

Managed Services

ATT&CK® Evaluations for Managed Services provides transparent and impartial insights into how managed security service providers (MSSPs) and managed detection and response (MDR) capabilities provide context of adversary behavior.

Industrial Control Systems (ICS)

MITRE Engenuity ATT&CK® Evaluations for Industrial Control Systems (ICS) clarify anomaly and threat detection capabilities of industrial control systems security solutions.

Trials

ATT&CK® Evaluation Trials allow more capabilities that do not fit in the aforementioned categories to be evaluated. The first Trial is for Deceptions.

Get Involved

Apply to be part of the ATT&CK Evaluations Community Advisory Board

Our commitment to innovation and consistency requires deeper interaction in a more structured format with the community we are looking to benefit. Our goal is to learn how we can make the data we present more accessible to the community at large. As an end user of ATT&CK Evaluations and a member of the infosec community, your insights are valuable to us.

Get In Touch

Learn more about how you can get involved with MITRE Engenuity.

Contact us

Stay informed.

Sign up for news about MITRE Engenuity and the critical technical challenges facing our nation and world.

Subscribe to updates