Beware: Clown-grade SOCs Still Abound

by Anton Chuvakin on December 12, 2019

I have written a lot about building and running Security Operations Centers (SOCs) in the past, and of course so did a lot of other smart people. However, I still hear stories of many abysmal failures with SOC planning and execution. And even of complete “cargo cult” SOCs out there …

Lately, my very favorite dumb mistake in a SOC has been “trying to do threat hunting before threat detection” and its truly idiotic cousin “trying to hunt before there is logging and log data collection.” IMHO, the latter truly deserves a clown-grade label!

Is this your clown-grade SOC during an incident? 🙂

Another sad favorite is about people who say “oh, yes, we have a SOC, and its name is <popular MSSP name>” Hmmm? While using an MSSP is a choice, it also happens to be a choice of NOT building a SOC (unless you run a hybrid setup of SOC and MDR, that is).

Admittedly, I still see plenty of “classic” SIEM-centric (but really, SIEM-exclusive) SOCs where the analysts lack any other visibility, but their SIEM. As a result, they only see logs. And only those logs their SIEM vendor wants them to see! Often, this is exacerbated by the fact that the line analysts have no access and even no influence on detection content. This means that they are overwhelmed by alerts while lacking any chance to tune them. Sad indeed!

Admittedly, running your SOC in 2020 like it was in 2002 is a mistake. Sole focus on the endless stream of SIEM alerts and no deeper analysis of what is going on across the extended enterprise (hence no real situational awareness) won’t make for a good SOC experience. Today’s good SOC does more than “disposes of” SIEM alerts.

Sponsorships Available

But guess what else is wrong? Running your SOC in 2020 as if 2002 never happened! And this is where Anton actually get angry, full-on, Russian bear angry. This is the situation where this whole “we hunt but we don’t detect” clownage comes in. This is where people endlessly search their meager log data for cryptic indicators (“OMG, let’s look for whether anybody RDP’d into our servers with keyboard country layout set to Iran”) without actually spending any time with more traditional detection approaches that work across a broader range of threats most organizations face.

So this is how you get to “cargo cult” SOC — I do see people with really low levels of security operations maturity try to mimic some of the practices from a really advanced SOC (literally, somebody who is 1 out of 5 copies random “ninja moves” from some organization that is a 6 of 5). This often relates to threat hunting, threat intel or (worse) custom security data lakes.

Note that years ago, my top favorite SOC mistake was the one where people will just relabel their NOC into SOC. I hear this happens less these days, because perhaps all barely-suitable NOC doors have been relabeled with SOC plaques already 🙂

The next one comes from the “SOC metrics that ruin SOCs” department. It is the classic mistake of focusing solely on alert triage speed/time. Let me ask you the obvious: if you make “close alerts faster” the main goal of your SOC, guess what will happen? Exactly. You too will have a chance to do this.

Now, a SOC is ultimately a team of people (this is a useful reminder to the delusional ones who say “yes, we can sell you a SOC today”). And people work well when they are happy. I’ve seen some nasty “SOC decay” cases where repetitive and manual work just killed off the talent. The resulting skeleton crew of leftovers proved no match to the best the attackers were able to throw at them. Not working to retain good people kills SOCs. Hence, back to this website.

Along the same line, I’ve seen SOCs that tried to automate tasks and processes to save the people from being burned by all the routine. Great? Well, they didn’t have any processes written down or even clearly represented in anybody’s head. Sure, there are tools (such as SOAR) that deliver value by automating, improving consistency and reducing mistakes, but to automate something you typically need to first have it in hand.

There you have it, its a bit of a rant, but hopefully it will let you avoid some of the mistakes in your SOC.

Beware: Clown-grade SOCs Still Abound was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.