Weighting of decision criteria

Many of you are familiar with multi-criteria (multi-attribute) scoring methods for evaluating the alternatives in a decision.  Before you can score alternatives (estimate their relative effectiveness against each factor), you should first set the relative priority of the criteria/factors.  Remember that these criteria are the way that the stakeholders in the decision express what they value; they certainly will value some factors more than others.  These values are typically called “weights” or “weighting factors”.  Weights express the relative importance of one factor compared to others, averaged across the inputs and experience of all the decision stakeholders.

I’ve mentioned before that I think of criteria as arrows (vectors) that point in the direction of “goodness”.  Before you attempt to assign relative weights to these criteria, you should first set 2 values along each arrow:

• Must limit: stakeholder walkaway point, the value at which you will reject an alternative from further consideration
• Ideal value: realistic, but stretch goal that you think any alternative might deliver

Setting these values before you attempt to prioritize the factors can help you avoid a common weighting trap.  It’s very common for folks to think of a factor with a very stringent, demanding must limit as one that is extremely important, even critical.  They will say, “it’s critical that I get a car that gets at least 40 miles per gallon”.  However, if you set your must limit that high, then it will usually imply that you should give the “Fuel economy” factor a fairly low weight relative to other factors.  That’s because you’re actually weighting the relative value of “margin” beyond your must limit; answering the question, “Assuming that any vehicle I buy gets at least 40 mpg, how valuable is fuel economy beyond that level relative to comfort margin, carrying capacity margin, acceleration margin, resale value margin, purchase cost margin, etc. ?”

I call this approach to weighting “relative margin valuation“; I’m sure it has a fancier name elsewhere in decision analysis literature, but I’m already approaching my Must limit for writing today’s blog entry …