Why This Paper? or Mysteries of Testing Security!

Some of you have been wondering why we decided to embark on a project that resulted in our paper called “Threat-Oriented Approaches to Test Security in Production” [Gartner GTP access required].

For sure, the same research project also produced our world-famous BAS paper, but this one is a more traditional here-is-a-new-tech-and-how-to-use-it kinda document.

The testing paper, as we called it during development, is different. While some may consider it “a WHAT paper” (as opposed to “HOW papers” such as our SIEM, VM or MSSP guidance documents), it does seek to popularize the concept and practice of testing security in production.

Look, we all secretly know that much / most of deployed security technologies and developed security processes are “faith-based” (or “luck-based” for those militant atheists). Some may counter that they are “risk-based” in their enlightened organization, but I’d counter-counter that your risks are also very often “faith-based” themselves. And even if you somehow “know” your risks, security technology and practice real-world effectiveness is so uncertain in most cases as to be purely an article of faith.

For god’s sake, after a quarter of a century of trying, we cannot agree on how to test anti-malware. In a lab. In a vacuum. With an unlimited supply of vacuum-breathing lab rats. Also, we debate whether changing passwords every 90 days has security value. For 50 years. With no resolution. In sight.

This is the world we live in. This is the world we’ve lived in since the dawn of the infosec era in the late 1980s. Calling it “cyber” does not help. So, let’s just quietly accept it and then seek to change it! Our paperis our attempt to initiate such change.

We think security professionals need to TEST MORE and BELIEVE LESS! You may say that security people are a skeptical bunch naturally, but – look – see those shiny security appliances? Who do you think bought them?!

So, we scoped the research to cover “tests used to verify the current status of production environments, obtaining evidence directly from them.” Also, we look at “tests are threat-focused, that is, they use methods or look for data used by threat actors during their attacks.” We believe that this is a good way to test security and to make it more fact-based, or evidence-based. This is bigger than BAS, and not the same as pentesting, even though both play a role.

There you have it! Now go and Test More! Gather data!! Make security fact-based!!! [again? no, not again – it never really was :-)]

The paper in question:

• Threat-Oriented Approaches to Test Security in Production [Gartner GTP access required]

Related blog posts:

• Our Threat Testing and BAS Papers Are Out!
• How Much of Your Security Gear Is Misconfigured or Not Configured?
• Security Testing: At What Level?
• On Negative Pressure or Why NOT Objectively Test Security?
• The Bane of All Security Tests: Acting on Results
• Threat Simulation – How real does it have to be? (by Augusto)
• New Research: How to Actually Test Security?
• Threat Simulation Call to Action for 2018

*** This is a Security Bloggers Network syndicated blog from Anton Chuvakin authored by Anton Chuvakin. Read the original post at: https://blogs.gartner.com/anton-chuvakin/2018/06/12/why-this-paper-or-mysteries-of-testing-security/