SBN

SOC Threat Coverage Analysis — Why/How?

SOC Threat Coverage Analysis — Why/How?

As I mentioned in Detection Coverage and Detection-in-Depth, the topic of threat detection coverage has long fascinated me. Back in my analyst days, we looked at it as a part of a security use case lifecycle process. For example, we focused on things like number and quality of alerts per SIEM use case, false/useless alert (“false positive”) numbers and ratios (to useful alerts), escalations to incident response, tuning, etc.

But what about a more comprehensive look at detection coverage inside each tool? Is there a way to assess the net threat coverage represented by the aggregate detection coverage inside each tool and then all tools? What about coverage analysis down to the rules presence, performance and quality?

A recent SIEM data analysis released by CardinalOps, a startup in the threat coverage space, suggests that the detection coverage gap is large at many organizations. Log source configuration errors, broken log collectors, insufficient breadth of rules, rule errors, noisy rules, and other factors contribute to poor coverage in the average organization. This got me thinking about this detection-in-depth and detection coverage again.

To an extent, some Breach and Attack Simulation (BAS) vendors try to go there. But ultimately this is NOT a job for a BAS that needs to retain its judge or arbiter role, rather than live deep inside the security operations machinery. Here we need something a lot more operational and a lot more plugged into the SOC tools and processes.

Also to an extent, MITRE ATT&CK (and especially CAR) would be helpful for this, but it is a framework, not a tool or even a methodology. You can MITRE content to map some of the threats to detections, but there are more things involved in obtaining and, especially, keeping your detection coverage map current.

So, how would I approach systematically looking at detection coverage in my SOC?

  1. Do I know what threats I want to detect? Threat assessment process will answer that.
  2. Do I know what detection content (rules, algorithms, models, etc) I need for this? Detection engineering or use case management process will lead to the right content here.
  3. Do I know what data I need to run the detection content on? Some form of log/data source optimization process helps.
  4. Am I collecting this data? A checklist can help here.
  5. Is the data being collected in the right format, being parsed correctly (if needed) to drive the detection content in question?
  6. Is the rule actually working in real life? Test automation process will reveal that.

A few of these needs to be run in a loop (“Is this working? Is this working after some system changes were made?”). This is a clear security engineering challenge and opportunity.

(P.S.: Related to this topic, I want to announce that I joined the Advisory Board of CardinalOps; they call this area Threat Coverage Optimization; they have a research report out)

Related blog posts:


SOC Threat Coverage Analysis — Why/How? was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

*** This is a Security Bloggers Network syndicated blog from Stories by Anton Chuvakin on Medium authored by Anton Chuvakin. Read the original post at: https://medium.com/anton-on-security/soc-threat-coverage-analysis-why-how-24055307528f?source=rss-11065c9e943e------2