New Research: How to Actually Test Security?

As I alluded here, we [Augusto and me] will be starting an epic new research project on testing security [BTW, should we codename it “Testing Security”, Augusto? :-)]

First, a quick poll: how many types of security testing do you know? Let me try…

  • Penetration testing (PT)
  • Red teaming (RT) – differences of PT and RT are discussed here
  • Vulnerability assessment
  • Application security testing (AST)
  • Security rating services (like BitSight and SecurityScorecard)
  • Attack, threat, breach simulation tools (details)
  • Others, possibly many others….

Now, at the risk of sounding too philosophical, what does it even mean “to test one’s security”? At this early stage of our effort, all bets are off, but we do want to look into testing of security technologies and processes (such as detection and response processes), with the focus on outcomes, not controls. In essence, we want to look into testing the effectiveness, not the presence, of security controls.

Please share how you think of TESTING SECURITY … and please don’t say that “a good pentest should be enough security testing for everybody.” :-)

Note that we will try to keep the focus of this on actually TESTING, not other forms of evaluating, measuring, assessing or guessing about security … Asking people how secure they think they are isn’t testing. Questionnaires isn’t testing. Reviewing policies isn’t testing. Even reviewing the lists of controls they think they deployed isn’t testing (IMHO).

In fact, overall, paper security isn’t.

Blog posts related to Testing Security project:

This is a Security Bloggers Network syndicated blog post authored by Anton Chuvakin. Read the original post at: Anton Chuvakin