Scoring alternatives - keep it simple

The evaluation of alternatives in a decision should be a simple process that unlocks many insights.  Effective scoring begins with a clear definition of criteria.  Each criterion (factor, stakeholder goal) represents an arrow or vector in the direction of “goodness”.  Scoring is the process of placing the alternative somewhere along this arrow based on its estimate performance.   For this to have meaning, you should define at least 2 points along the line to set up a scoring scale:

• Must limit: The value (of the performance estimate) at which key stakeholders will walk away from any alternative and reject it from further consideration; their minimum requirement for success.
• Ideal value: The best value (of the performance estimate) that you believe any real-world alternative could deliver; a stretch value that aims very high, but is still possible.

The Must limit is usually assigned a score of 0 because it provides no margin (”wiggle room”) for the stakeholders.  The Ideal value represents the “perfect 10″.

This type of scoring scale can be used for both quantitative (cost, power consumption, time to market) and qualitative factors (strategic fit, differentiation, quality of life).

If you wanted to be more rigorous in your thinking, you could build a utility function that maps your performance estimates to a score (that represents utility, aka value delivered) for each criterion.  However, when facing a large number of decisions in a new business or technology area, you’ll probably lack trusted data from which to generate a formal utility function for most criteria.  I find using a simplified scoring scale based on scoring bands to be adequate.

An “almost linear” scoring curve works well, with a “point of diminishing returns” (the hinge point on the curve) reached as you approach the ideal value.

Don’t underestimate the insights that you can gain when doing this scoring; it’s not just a mechanical, number-crunching process.  You apply judgment when you distinguish between the effectiveness of alternatives.  Simply recording a few “notes to self” that explain the rationale for your scoring will crystallize the definition of the alternatives in your mind.  This can create a deeper understanding of the structure and expected behavior of alternatives, leading to opportunities for optimization and early identification of implementation tasks, risks to mitigate and derived requirements.