Can We Have “Detection as Code”?

by Anton Chuvakin on September 21, 2020

One more idea that has been bugging me for years is an idea of “detection as code.” Why is it bugging me and why should anybody else care?

First, is “detection as code” just a glamorous term for what you did when you loaded your Snort rules in cvs in, say, 1999? Well, not exactly.

What I mean by “detection as code” is a more systematic, flexible and comprehensive approach to threat detection that is somewhat inspired by software development (hence the “as code” tag). Just as infrastructure as code (IaC) is not merely about treating your little shell scripts as real software, but about machine-readable definition files and descriptive models for infrastructure.

Why do we need this concept? This is a good question! Historically, from the days of first IDS (1987) to the sad days of “IDS is dead” (2003) and then to today, detection got a bit of a bad reputation. We can debate this, to be sure, but most would probably agree that threat detection never “grew up” to be a systematic discipline, with productive automation and predictable (and predictably good!) results. In fact, some would say that “Your detections aren’t working.” And this is after ~35 years of trying …

Detection engineering is a set of practices and systems to deliver modern and effective threat detection. Done right, it can change security operations just as DevOps changed the stolid world of “IT management.” You basically want to devops (yes, I made it a word) your detection engineering. I think “detection as code” is a cool name for this shift!

Sponsorships Available

As you see, this is not so much about treating detections as code, but about growing detection engineering to be a “real” practice, built on modern principles used elsewhere in IT (agile this, or DevOps whatever).

Now, to hunt for the true top-tier APTs, you probably need to be an artist, not merely a great security engineer (IMHO, best threat hunting is both art and science, and frankly more art than science….). But even here, to enable “artistic” creativity in solving threat detection problems we need to make sure those solutions function on a predictable layer. Moreover, for many other detection pursuits, such as detecting ransomware early, we mostly need automated, systematic, repeatable, predictable and shareable approaches.

OK, how do we do “detection as code”? How would I describe the characteristics of this approach?

Detection content versioning so that you can truly understand what specific rule or model triggered an alert — even if this alert was last July. This is even more important if you use a mix of real-time and historical detections.
Proper “QA” for detection content that covers both testing for broken alerts (such as those that never fire or those that won’t fire when the intended threat materializes, and of course those that fire where there is no threat) and testing for gaps in detection overall. “False positives” handling, naturally, get thrown into this chute as well.
Content (code) reuse and modularity of detection content, as well as community sharing of content, just as it happens for real programming languages (I suspect this is what my esteemed colleague describes here). As a reminder, detection content does not equal rules; but covers rules, signatures, analytics, algorithms, etc.
Cross-vendor content would be nice, after all we don’t really program in “vendor X python” or “big company C” (even though we used to), we just write in C or Python. In the detection realm, we have Sigma and YARA (and YARA-L too). We have ATT&CK too, but this is more about organizing content, not cross-vendor writing of the content today.
I also think that getting to cross-tool detection content would be great, wherever possible. For example, you can look for a hash in EDR data and also in NDR; and in logs as well. SIEM alone won’t do.
Metrics and improvement are also key; the above items will give you plenty of metrics (from coverage to failure rates), but it is up to you to structure this process so that you get better.
While you may not be looking at building a full CI/CD pipeline for detections to continuously build, refine, deploy and run detection logic in whatever product(s), I’ve met people who did just that. To me, these people really practice detection as code.
Finally, I don’t really think this means that your detections need to be expressed in a programming language (like Python here and here or Jupyter notebooks). What matters to me is the approach and thinking, not actual code (but we can have this debate later, if somebody insists)

Anything else I missed?

For our recent SANS paper / webcast, that mentioned this topic, we crafted this example visual:

Source: recent SANS paper.

Finally, let’s cattle-prod the elephant in the room: what about the crowd that just does not want anything “as code”? They also don’t like to create their own detections at all. In fact, they like their detections as easy as pushing an ON button or downloading a detection pack from a vendor? This is fine.

Personally, I’ve met enough security people who run away screaming from any technology that is “too flexible”, “very configurable” and even “programmable” (or: “… as code”) because their past experience indicates that this just means failure (at their organization). However, to detect, you need both a tool and content. Hence, both will have to come from somewhere: you can build, buy, rent, but you must pick.

Now, upon reading this, some of you may say “duh … what is not painfully obvious about it?” but I can assure you most people in the security industry do NOT think like that. In fact, such thinking is alien to most, in my experience. Maybe they think detection is a product feature. Or perhaps they think that detection is some magical “threat” content that comes from “the cloud.”

Hence, “detection as code” is not really an approach change for them, but a more philosophical upheaval. Still, I foresee that threat detection will always be a healthy mix of both an engineering and a creative pursuit….

Thoughts?

P.S. Thanks to Brandon Levene for hugely useful contributions to this thinking!

Can We Have “Detection as Code”? was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.