The Evaluation: Stage 3 – Execution of Testing

EH-Net - Los - The Evaluation: Stage 3 - Execution of TestingWelcome back. In the last few articles we discussed how to set up an evaluation that’s both defensible & provides value to your immediate needs, and, maybe, more clearly defined what good pizza is. In this edition, I’ll address the importance of the process of the execution of testing.

We’ve already talked through defining the problem, and the success criteria. Now it’s time to actually run tests that will give you results you can defend and live with. The good news is, there are rules that govern this sort of thing, and they’re pretty good. I’ll break the sub-components of testing down into three parts: define the testing framework, execute with consistency, trust the results. Let’s talk through those here.

Define the Testing Framework

The key to testing is making it fair and equitable. Everyone you bring to the table should have a fair shot of winning your approval within the rules of engagement. Whoops! I almost forgot Rules of Engagement.

So Step 1, defining Rules of Engagement. When you bring 3 widgets into a test, what will the rules be? Will you work off of a stock configuration? Will you allow tuning? Will the participants be given knowledge of the testing environment? How much knowledge? What happens when something fails, does the participant get a tune and retry? These and a billion more are the right questions to answer ahead of time. Everyone in your evaluation should have to play by the same rules, or the thing is a farce.

So now that you have your rules of engagement, it’s time to build a test harness. If you’re evaluating a network-based intrusion prevention platform, you’ll want to make sure you have a mock network segment designed that someone can plug into that accurately mirrors your real production environment. Otherwise, (Read more...)

*** This is a Security Bloggers Network syndicated blog from The Ethical Hacker Network authored by Rafal Los. Read the original post at: