ATT&CK Evaluations: Managed Services Inaugural Round of Results Now Available

        • Who We Are
        • Learn more about MITRE Engenuity’s journey as a hub for transformative innovation.

        • How We Engage
        • We forge innovative partnerships to generate whole-of-nation solutions to complex technological problems.

        • Contact Us
        • Connect with a member of the MITRE Engenuity team and ensure your inquiry gets to the right people.

        • Semiconductors
        • Dive into the revolutionary work that MITRE Engenuity is doing within this critical ecosystem.

        • Circuit Talk
        • Hear directly from the semiconductor experts through our speaker series featuring titans of industry, groundbreaking researchers, and many more.

        • Cybersecurity
        • We are relentlessly advancing the art of threat-informed defense, anchored by a belief that we can improve our defenses with a systemic application of a deep understanding of adversary tradecraft and technology.

        • ATT&CK Evaluations
        • We offer objective analysis of cyber products and features – see our latest results.

        • Center for Threat-Informed Defense
        • Read more about the cutting-edge research and development being done with input from our participant organizations, featuring some of the top security operations centers.

        • MITRE ATT&CK Defender
        • Strengthen your threat-informed defense capabilities with our cybersecurity trainings taught by MITRE ATT&CK subject matter experts.

        • Telecom
        • The transformative power of 5G shifts paradigms across industries and empowers businesses to change the way they interact with people. See how MITRE Engenuity is impacting the next generation of telecommunications. 

        • Open Generation 5G Consortium
        • We are getting to our 5G future faster. Discover how we are accelerating network technology and device-to-device application innovation through use case-focused R&D in the Open Generation 5G Consortium.

        • Health
        • We identify potential health security threats to ensure faster public health pandemic responses and incubate new ideas to ensure national health security.

        • Growing Impact
        • We deliver positive public impact through advanced technological innovation projects.

        • Cyber Risk Model for Mobile Digital Financial Services: Securing Mobile Money Services. Explore Our Cyber Risk Model for Mobile Financial Services product

ATT&CK EVALUATIONS:

Using Evaluations

How To Use & Extract Value from MITRE Engenuity ATT&CK® Evaluations

Knowing how to review ATT&CK Evaluations best empowers you to pick the vendor that is most relevant to your organization’s needs and helps you understand your tool. Many people stop their analyses of ATT&CK Evaluations at the summary, but it’s important to go beyond because:

  • Each section informs the other for a meaningful analysis.
  • The sum of what you discover in the details is greater than the whole found in single-number results.

Remember, we don’t compare or rate providers or tools. We only evaluate individual tools.

Drone flying under sunset sky

Tips To Best Understand & Leverage ATT&CK Evaluations

ATT&CK Evaluations are a starting point.

We use an open-book and minimally sized environment to understand baseline capabilities of solutions. Operationalization of these solutions is important to consider in the context of your organization, including false positive generation.

There are no winners.

The goal of ATT&CK Evaluations is to show the different capabilities of each provider.

Not all techniques are created equal.

A technique detection for credential dumping may not have the same value as a technique for process discovery due to the severity of the actions. The category gives you a general idea, but you should dive into the details to understand the technique and detection.

Look at the UI/UX view for more than aesthetics.

Consider how the UI/UX would work, not just the look and feel of it. Are you seeing the information you’d need in real-time?

Determine how a provider’s tool presents itself to your analysts.

Thumb through provider screenshots to see if you’re being presented the data you need and review a tool’s overall usability. Is it easy to find the data you want to see in real-time?

 

See if a tool detects and prioritizes known threats to your organization.

Understanding a tool’s detection and prioritization capabilities will help you know if you’ll get an alert at the right time and in a way that allows you to trigger effective deterrence.

 

Understand the data sources: Are they effective?

For example, from what data source was a detection triggered, and how does that trigger fit into your cybersecurity needs and tactics?

 

KEY PARTS OF AN ENTERPRISE ATT&CK EVALUATION

  • Evaluation Overview Page

    Each Enterprise evaluation has available information specific to those evaluations. The overview page for each round provides high-level information and includes:

    • Links to the results
    • A list of participants
    • A description of the adversary
    • Emulation notes
    • The technique scope
    • Environment notes
    • Detection categories
    • Related resources about the evaluation
    Access an Enterprise Evaluation Overview
  • Detection Summary Screenshots

    A detection summary shows a collection of screenshots from the provider. This allows users to quickly review the UX/UI of the tool.

  • Detailed Results & Screenshots

    Within Evaluations results for every provider, there are specifics on each scenario and details for every step and sub-step of the emulation plan including:

    • Tactic, technique, and sub-technique information
    • Detection categories
    • Detection criteria and data sources
    • Screen shots of detections within the provider’s environment
  • Protection Results

    These are linear tests with a defined start and end. They are opt-in and not every result includes them.

  • Participant Configuration

    This includes provider product versions and descriptions used in the evaluation.

  • JSON (Java Script Object Notation) File

    The JSON file can be downloaded to dive more deeply into results and build internal analytics to further parse data.

Just Released

Our first Managed Services Evaluation Just released

ATT&CK Evaluations just released its inaugural Managed Services evaluations. The evaluation results from 16 participating managed services providers are now available.

attack flow project

Sign up to receive the guide to using the new ATT&CK Evaluations Managed Services Results:

Evaluations by Industry

The ATT&CK Evaluations program continues to develop new methodologies, and open new rounds of evaluations. Currently, there are four types of ongoing ATT&CK Evaluations available:

Enterprise

ATT&CK® Evaluation for Enterprise empowers end-users to make more informed decisions on endpoint detection capabilities by articulating how each vendor can protect against or detect adversary behavior.

Managed Services

ATT&CK® Evaluations for Managed Services provides transparent and impartial insights into how managed security service providers (MSSPs) and managed detection and response (MDR) capabilities provide context of adversary behavior.

Industrial Control Systems (ICS)

MITRE Engenuity ATT&CK® Evaluations for Industrial Control Systems (ICS) clarify anomaly and threat detection capabilities of industrial control systems security solutions.

Trials

ATT&CK® Evaluation Trials allow more capabilities that do not fit in the aforementioned categories to be evaluated. The first Trial is for Deceptions.

Get Involved

Apply to be part of the ATT&CK Evaluations Community Advisory Board

Our commitment to innovation and consistency requires deeper interaction in a more structured format with the community we are looking to benefit. Our goal is to learn how we can make the data we present more accessible to the community at large. As an end user of ATT&CK Evaluations and a member of the infosec community, your insights are valuable to us.

Get In Touch

Learn more about how you can get involved with MITRE Engenuity.

Contact us

Stay informed.

Sign up for news about MITRE Engenuity and the critical technical challenges facing our nation and world.

Subscribe to updates