The Evaluation – Four Phases to Finding “Better” Solutions

EH-Net - Los - The Evaluation - Top Gun PicWhether you’ve spent your career in cyber security on the vendor/provider side or the enterprise side of the table, you’ve no doubt participated in the circus that is the ‘evaluation’. Whether you’re the buyer trying to make a smart purchase, or the seller trying to make a smart sale, the evaluation is a fact of life.

That said, evaluations are one of the most difficult parts of the role. I often refer to the process using a term borrowed from Ben Kepes, a “goat rodeo”. Evaluations come in four stages from my experience. Stage 1 is the definition of the problem to be solved. Stage 2 is defining the criteria. Stage 3 is executing testing. Stage 4 is determining the outcome. I’ll give a brief overview of each stage here and set up for a five-part series starting with this article that will detail the challenges and strategies for overcoming them.

Evaluation Stage 1 – Definition of the Problem to be Solved

Many technology and service evaluations perish before they get out of the starting blocks. The main cause, every time, is poor definition of the problem to solve. If you have the word “better” in your problem statement you’re likely going down this path. I’m going to assume you’ve written a problem statement. If you haven’t, that is step #1.

A problem statement must identify what’s deficient, why you believe it’s deficient, and by how much it needs to improve. Be specific! A problem statement should also provide concrete evidence of the deficiency. If you believe that your current SIEM is insufficient for your organization’s needs, you should know why. Perhaps it’s unable to keep up with the volume of events that feed into it for analysis. Perhaps it’s missing some critical ability to parse or ingest data. Perhaps (Read more...)

*** This is a Security Bloggers Network syndicated blog from The Ethical Hacker Network authored by Rafal Los. Read the original post at: