By Michael Mumcuoglu, CEO & Co-Founder, CardinalOps
As we approach the second half of a year punctuated by ransomware and supply chain attacks, a top concern on nearly everyone’s mind is security budgets. A closely-related topic is management-level reporting. With strong economic headwinds, how do we effectively report our security posture to executives and boards in order to demonstrate effective use of our limited resources?
A big part of this is rethinking how security executives approach reporting. Typically, the report to the board has been around metrics like mean time to detect (MTTD) and mean time to respond (MTTR). However, MTTD and MTTR metrics only describe how good your team is at responding to attacks after you have detected them, but they’re missing critical information about which attacks were never – and will never be – detected in the first place.
These missed attacks often stem from either hidden gaps in detection coverage — or due to alerts that got buried in a sea of noisy alerts and were never even pursued by the Security Operations Center (SOC) team.
According to IDC, 20-30% of all alerts are simply ignored or not investigated in a timely manner, frequently due to classic “alert fatigue” caused by too many noisy alerts.
Another disadvantage of MTTD and MTTR metrics is that they don’t give management an accurate representation of risk to the business. Instead, we should be looking at metrics that describe the organization’s readiness to detect Tactics, Techniques, and Procedures (TTPs) that target business-critical systems such as cloud applications, or crown jewel assets such as databases with PII and other sensitive data. In other words, we need to be able to report on the organization’s detection posture.
Why prevention is insufficient
A key tenet of security is that you cannot effectively prevent all attacks. The current thinking is that our mindset needs to shift from prevention to rapid detection and response. In fact, according to Dr. Eric Cole, a well-known SANS Fellow and security consultant, prevention is ideal, but detection is a must.
Our constantly-expanding attack surface is part of the challenge. One report found that enterprise cyber assets have increased by 133 percent year-on-year, from an average of 165,000 in 2022 to 393,419 in 2023. With that many assets to defend – including cloud assets like containers that don’t even support EDR agents – you are setting yourself up for failure by trying to prevent every attack. But where do you begin?
Following the roadmap
Enter the MITRE ATT&CK framework. The framework extends the traditional intrusion kill chain model to go beyond IOCs (like IP addresses, which attackers can change constantly) in order to catalog all known adversary playbooks and behaviors (TTPs).
As the standard framework for understanding adversary behavior, MITRE ATT&CK now describes more than 500 techniques and sub-techniques used by threat groups such as APT28, the Lazarus Group, FIN7, and LAPSUS$.
According to ESG research, 89% of organizations currently use MITRE ATT&CK to reduce risk for security operations use cases such as determining priorities for detection engineering, applying threat intelligence to alert triage, and gaining a better understanding of adversary TTPs.
Another advantage of MITRE ATT&CK is that it provides a common language to communicate about attack behaviors across internal security teams (threat hunters, red teams, detection engineering, etc.) as well as across organizations (like ISACs).
As a result, tracking MITRE ATT&CK coverage is an ideal metric to track and report on your organization’s detection posture.
The inherent challenges
Despite the benefits of MITRE ATT&CK, many organizations find it challenging to measure their detection coverage and address the highest-priority coverage gaps that can lead to breaches.
In fact, based on our data-driven research analyzing more than 4,000 rules across diverse SIEM platforms in production environments — including Splunk, Microsoft Sentinel, IBM QRadar, and Sumo Logic – enterprise SIEMs are typically missing detections for 76% of all MITRE ATT&CK techniques used by adversaries. Put another way, using MITRE ATT&CK v13 as the baseline, they are blind to around 150 techniques used by adversaries.
Is it lack of caring that prevents organizations from ensuring they have the right detections in their SIEMs? Absolutely not. The simple truth is that effectively managing SIEMs is incredibly complex. New log sources are constantly being added and detection engineers find themselves struggling to keep up with the latest vulnerabilities and changes in their attack surface. Plus they constantly find themselves scrambling in a reactive mode after successfully being attacked by Red teams and penetration testers.
These challenges are compounded by the biggest challenge: finding and retaining skilled detection engineers, especially when organizations are at the same time adopting newer SIEMs – such as cloud-native SIEMs with unfamiliar query languages – to reduce data ingestion costs.
What needs to happen: focus on streamlining detection engineering processes
Automation is widely-accepted as a top priority for improving the effectiveness of the SOC, but until now it has only been applied to other areas besides detection engineering, such as incident response (with SOAR) and anomaly detection (with behavioral analytics).
In fact, in most organizations, detection engineering tends to be based on highly-manual processes, tribal knowledge, and individual “ninjas” rather than formal, documented workflows enabled by automation.
For example, security teams are often required to manually map detections to MITRE ATT&CK using spreadsheets, which is time consuming and error-prone. And they are responsible for manually identifying existing detections that are broken or misconfigured, due to missing telemetry or other data quality issues, for example (in fact, our research found that on average, 12% of existing detections in production SIEMs are broken and will never fire). Finally they are also responsible for continuously researching the latest exploits and manually developing high-fidelity detections for them.
These are not tasks that require the creativity of a human. In fact, automation is better at these kinds of tasks that are tedious and exhausting for a human practitioner.
Despite the buzz around AI created by ChatGPT, which can create impressive answers to a wide range of questions, automation isn’t a silver bullet – but it also shouldn’t be discounted either. The future of security is found in the marriage of automation and human creativity. Security leaders will benefit greatly from freeing their security professionals to think creatively and focus on more complex and interesting challenges – such as threat hunting and understanding new and novel attack behaviors – rather than mundane tasks related to managing their SIEMs and tracking their MITRE ATT&CK detection coverage.
Once this is accomplished, CISOs will be well-equipped to answer strategic questions such as “How prepared are we to detect the latest high-priority threats?” and “What is the roadmap for improving our detection posture over time?” And because they will not only be using standard metrics in their response – the metrics will also be achievable, predictable, and based on a thoughtful, threat-informed strategy.
About the Author
Michael Mumcuoglu is the CEO and Co-Founder of detection posture management company CardinalOps. He is a serial entrepreneur that is passionate about technology, cybersecurity, and leadership. Prior to CardinalOps, Michael co-founded LightCyber, a pioneer in behavioral attack detection acquired by Palo Alto Networks (NYSE: PANW) in 2017, where he served as Vice President of Engineering for the Cortex XDR platform. Prior to founding LightCyber and other startups, Michael served in various cybersecurity roles in an elite intelligence division of the Israel Defense Forces.
Michael can be reached online at @xmichaelm and to learn more about CardinalOps, visit https://cardinalops.com/ or follow at @CardinalOps.
;
We are in our 11th year, and Global InfoSec Awards are incredibly well received – helping build buzz, customer awareness, sales and marketing growth opportunities, investment opportunities and so much more.