The Evaluation: Stage 4 – Determining the Outcome

Wrapping up this series on evaluating technologies and services is actually fairly simple. Determining the outcome of an evaluation is, in fact, mechanical. That is to say, that you shouldn’t have any emotion or subjectivity in your determination of the outcome.

When I think back at all the evals I’ve led or participated in over my 20+ years (damn, I’m getting old… get off my lawn), I can separate them all into two categories: those that went well and those that didn’t. If I’m truly honest with myself, the reason for the difference is fair and easy to see. That difference is  subjectivity. The moment you add feelings into an evaluation you’ve lost.

Sponsorships Available

Outcome How To

So how do you do evaluations without feelings? Spreadsheets, of course.

Set up your spreadsheet with a table of 4 columns, and as many rows as you have evaluation criteria. Now you’ll remember that in The Evaluation: Stage 2 – Definition of Success Criteria (the one with the Lou Malnatti’s pizza example), I told you that you should have an odd number of success criteria. What I didn’t tell you is how many you should have… the answer here is “enough”. Actually anything more than 11 (remember, odd numbers) is good.

So column 1 is the name of the success criteria.

Column 2 is the weight; you’ll use 1, 10, 100 for “nice to have”, “should have”, and “must have” respectively. So for criteria that are the “can’t do withouts” – the ones that support the core of the use-case you’re trying to solve – you give that a 100, and so on.

Column 3 is the binary (pass/fail) metric and in that column you either give a 1 or 0. Nope, not 1 through 5 … 1 or 0.

Column (Read more...)