Multi-Criteria Decision Analysis (MCDA)

Purpose

Systematically evaluate alternatives or score entities (needs, concepts, portfolio items) against multiple weighted criteria, producing quantitative composite scores that support transparent, defensible decisions. MCDA replaces intuitive “gut-feel” judgement with structured reasoning while preserving expert judgement within a disciplined framework.

MCDA does not eliminate subjectivity—scoring individual criteria still requires human assessment. Its value lies in decomposition (breaking complex judgements into assessable dimensions), transparency (making weights and scores explicit and debatable), and consistency (applying the same framework across multiple evaluations, enabling fair comparison).

When to use

• Decision involves ≥3 evaluation criteria that may pull in different directions (trade-offs exist)
• Multiple stakeholders must agree on a decision and need shared reasoning framework
• Decisions must be documented and defensible (governance review, audit trail)
• Comparing or ranking ≥2 alternatives (concepts, needs, portfolio items)
• Scoring a single entity against a threshold (strategic fit, need attractiveness)

Output

Completed MCDA scorecard with:

1. Criteria × scores table with evidence column
2. Weighted composite score
3. Threshold-based category assignment
4. Sensitivity analysis summary
5. Scoring confidence level

For A1.4 specifically: two scorecards (Strategic Fit and Need Attractiveness) feeding into the P/D/D Decision Logic (the referenced method).

Inputs

• Decision context: what is being evaluated and what decision the score informs
• Criteria set: 3–7 criteria (fewer = insufficient decomposition; more = cognitive overload and spurious precision)
• Weight calibration source: organisational strategy documents (B1), portfolio policy (I1), or stakeholder consensus workshop
• Evidence base per criterion: data, research, expert assessments that support scoring
• Threshold definitions: pre-set boundaries mapping score ranges to decision categories

Complete procedure

Phase~1: Design the scorecard (before evaluation).

1. Define criteria (3–7). itemize
2. Source criteria from the decision context. For A1.4 strategic fit: B1 strategic objectives, B2 capability assessment, I1 portfolio policy. For A1.4 attractiveness: A1.2 validation data, market research.
3. Test independence: does scoring high on Criterion~A automatically produce a high score on Criterion~B? If yes, they overlap—merge or remove one.
4. Test completeness: could an entity score HIGH on all criteria yet still be a poor decision? If yes, a criterion is missing.
5. Test assessability: can each criterion be scored with available evidence? If not, either gather additional evidence or replace with a proxy criterion. itemize
6. Assign weights. Weights reflect “what matters most” for this specific decision context. Three approaches, from simplest to most rigorous: description[nosep]
7. [Direct assignment (fastest):] Decision-maker or team allocates 100 percentage points across criteria based on strategic judgement. Most common in practice. Risk: anchoring to first allocation. Mitigation: have 2–3 stakeholders assign independently, then average and discuss divergences.
8. [Pairwise comparison (moderate rigour):] For each pair of criteria, ask: “Which is more important, and by how much?” (1 = equal, 2 = slightly more, 3 = much more). Derive weights from comparison matrix. Based on Saaty's Analytic Hierarchy Process (AHP). Use when: ≥5 criteria make direct assignment cognitively difficult; stakeholders disagree on relative importance.
9. [Organisational calibration (most rigorous):] B5 Operations establishes standard weights reflecting B1 strategic priorities, reviewed annually. All A1.4 explorations use the same calibrated weights unless I2 approves deviation. Use when: organisation conducts ≥5 MCDA evaluations per year and needs cross-evaluation comparability. description A1.4 default weights (from the referenced method): TABLE0
10. Define anchored scales. For each criterion, write explicit definitions for every scale level (0, 1, 2, 3). Anchors should be: itemize
11. Observable: assessable from available evidence
12. Ordered: each level clearly “more” than the previous
13. Mutually exclusive: an entity fits exactly one level
14. Exhaustive: every plausible state maps to a level itemize Example (Strategic Priority Alignment): quote 0 = No alignment with any B1 strategic objective. 1 = Weak/indirect alignment; need is tangentially related to a secondary B1 objective. 2 = Moderate alignment; need directly supports a secondary B1 objective or indirectly supports a top-3 objective. 3 = Strong alignment; need directly supports a top-3 B1 strategic objective. quote
15. Set thresholds before scoring. Define score ranges that map to decision categories. Thresholds must be documented and agreed before any entity is scored. itemize
16. Strategic fit: ≥11 HIGH, 8–11 MEDIUM, <8 LOW.
17. Attractiveness: ≥9 HIGH, 6–8 MODERATE, <6 LOW. itemize Threshold-setting is a strategic act: lower thresholds advance more needs (volume), higher thresholds advance fewer (selectivity). B1 and I1 should calibrate thresholds to portfolio capacity.

Phase~2: Score the entity.

1. Score each criterion with evidence. For each criterion: enumerate[label=(*)]
2. Review the anchored scale definitions.
3. Assemble evidence (A1.2 data, B1 documents, market research, expert assessments).
4. Assign a score (0–3) by matching evidence to the anchor that best fits.
5. Document the evidence supporting the score in 1–3 sentences. This is mandatory. A score without evidence is an opinion, not an assessment. enumerate Half-scores. When evidence falls clearly between two anchors, half-scores (e.g., 2.5) are acceptable. Use sparingly: more than 2 half-scores per scorecard suggests anchors need refinement.
6. Calculate composite score. Apply the weighted-sum formula. For A1.4 Strategic Fit (5 criteria, 0–3 scale, weights summing to 100%): DM: S_fit = (s&lt;sub&gt;1&lt;/sub&gt; × 0.30) + (s&lt;sub&gt;2&lt;/sub&gt; × 0.20) + (s&lt;sub&gt;3&lt;/sub&gt; × 0.25) + (s&lt;sub&gt;4&lt;/sub&gt; × 0.15) + (s&lt;sub&gt;5&lt;/sub&gt; × 0.10) Maximum: 3 × (0.30 + 0.20 + 0.25 + 0.15 + 0.10) = 3 × 1.0 = 3.0. Wait—the A1.4 scorecards report maximums of 15.0 and 12.0. This is because A1.4 presents unscaled weighted sums (weight expressed as percentage points, not fractions): DM: S_fit = (s&lt;sub&gt;1&lt;/sub&gt; × 30%) + (s&lt;sub&gt;2&lt;/sub&gt; × 20%) + with each s_i [0,3] and weights as percentages, producing S [0, 15.0]. Either notation is valid; be consistent within the organisation.
7. Apply threshold. Map composite score to decision category (HIGH / MEDIUM / LOW or HIGH / MODERATE / LOW).

Phase~3: Validate and stress-test.

1. Conduct sensitivity analysis. Test whether the recommendation changes under plausible alternative assumptions: itemize
2. Weight sensitivity: Shift 10% from the highest-weighted criterion to the lowest-weighted. Does the threshold category change? If yes, the result is weight-sensitive — flag for I2 discussion.
3. Score sensitivity: For each criterion scored at a boundary (e.g., “either 2 or 3 is defensible”), test both scores. Does the composite cross a threshold? If yes, the result is score-sensitive on that criterion—gather additional evidence or flag uncertainty.
4. Threshold sensitivity: If the composite score is within 1~point of a threshold boundary, note the proximity. Scores of 10.5/15 (near the 11.0 HIGH boundary) deserve more scrutiny than scores of 13.5/15. itemize
5. Seek independent scoring (recommended for high-stakes decisions). Have a second assessor score independently using the same scorecard. Compare scores. itemize
6. Agreement within 0.5 per criterion: adequate inter-rater consistency.
7. Disagreement >1.0 on any criterion: discuss evidence, re-examine anchors, converge on justified score. If unresolvable, escalate to I2 with both scores and rationale. itemize
8. Document the complete scorecard. Final deliverable per MCDA application: itemize
9. Scorecard table (criteria, weights, scores, evidence per criterion)
10. Composite score and threshold category
11. Sensitivity analysis results (which variations, if any, change the outcome)
12. Scoring confidence: HIGH (all criteria well-evidenced, no boundary scores) / MEDIUM (1–2 criteria at boundary, evidence adequate) / LOW (multiple criteria poorly evidenced—flag for additional research) itemize

Quality criteria

1. Criteria independence: No double-counting. If “strategic alignment” and “portfolio fit” overlap heavily, merge or differentiate clearly.
2. Weights set before scoring: Weights reflect organisational values, not post-hoc manipulation to achieve desired outcome.
3. Anchored scales: Every scale level has a written definition. “3 = good” is insufficient.
4. Evidence-based scores: Every score has 1–3 sentences of supporting evidence citing specific sources (B1 document, A1.2 data point, market report).
5. Thresholds pre-set: Category boundaries documented before scoring begins.
6. Sensitivity analysis performed: Weight and score sensitivity tested for all high-stakes decisions.
7. Scoring confidence calibrated: Not uniformly “HIGH.”

When NOT to use

• Decision has a single dominant criterion (e.g., regulatory compliance is binary pass/fail—no weighting needed)
• Only 1–2 criteria matter and the answer is obvious—MCDA adds overhead without insight
• Data quality is too poor to score meaningfully (garbage in, garbage out with mathematical veneer)
• Decision is fundamentally political/strategic and quantification would create false objectivity

Core concepts

description[nosep] [Criterion:] A single evaluable dimension (e.g., “strategic priority alignment,” “need intensity,” “technical feasibility”). Good criteria are independent (non-overlapping), complete (covering all relevant dimensions), and assessable (evidence can support scoring).

[Weight:] Relative importance of each criterion, expressed as a percentage of total. Weights sum to 100%. Weights encode organisational values: “What matters most to us in this decision?”

[Scale:] The scoring range per criterion. This book uses a 0–3 ordinal scale throughout A1.4: 0 = absent/none, 1 = weak/low, 2 = moderate/partial, 3 = strong/high. Each level has an anchored definition specific to the criterion.

[Anchor:] A concrete description of what each scale level means for a specific criterion. Anchors prevent scoring drift and enable inter-rater consistency. Example: “Capability fit: 3 = strong existing capabilities; 2 = partial capabilities, addressable gaps; 1 = significant gaps, build from scratch; 0 = no relevant capabilities.”

[Composite score:] Weighted sum of criterion scores:

where w_i is the weight (as fraction) and s_i is the score for criterion~i.

[Threshold:] A pre-defined composite score boundary that maps to a decision category (e.g., ≥11/15 = HIGH fit Pursue signal; <8/15 = LOW fit Decline signal). Thresholds must be set before scoring to prevent post-hoc rationalisation. description

Variations by activity

When applying MCDA in activities other than A1.4, redesign the scorecard (criteria, weights, anchors, thresholds) for that decision context. Do not reuse A1.4 scorecards for concept selection or portfolio prioritisation — different decisions require different criteria.

Common errors

Mathematical note: Compensatory vs. non-compensatory MCDA. The weighted-sum approach used in A1.4 is compensatory: a low score on one criterion can be offset by a high score on another. This is appropriate when trade-offs are acceptable (moderate strategic fit compensated by exceptional attractiveness).

For decisions where certain criteria are non-negotiable (must-pass), apply a non-compensatory pre-filter before the weighted sum. In A1.4, feasibility serves this role: infeasibility triggers Decline regardless of fit or attractiveness scores. The P/D/D Decision Logic (the referenced method) implements this by evaluating feasibility as a gating criterion before applying the compensatory scorecards.

Organisations may designate additional non-compensatory criteria. Example: “Any need scoring 0/3 on strategic priority alignment is automatically Declined regardless of composite score.” Document such rules in the scorecard design phase.

Theoretical basis

• Keeney, R.~L. & Raiffa, H. (1976). Decisions with Multiple Objectives: Preferences and Value Tradeoffs. Foundational MCDA theory: multi-attribute utility functions, independence axioms, weight elicitation.
• Saaty, T.~L. (1980). The Analytic Hierarchy Process. Pairwise comparison method for weight derivation; consistency ratio for validating weight coherence.
• Belton, V. & Stewart, T.~J. (2002). Multiple Criteria Decision Analysis: An Integrated Approach. Comprehensive overview of MCDA variants; guidance on method selection.
• Goodwin, P. & Wright, G. (2014). Decision Analysis for Management Judgment (5th ed.). Practical applications of MCDA in managerial contexts; behavioural pitfalls.

Relationship to other methods

• Strategic Fit Scorecard (the referenced method) and Need Attractiveness Scorecard (the referenced method): specific MCDA instantiations with pre-defined criteria, weights, and anchors for A1.4.
• P/D/D Decision Logic (the referenced method): consumes MCDA outputs (two composite scores) as inputs to the four-factor decision tree.
• Need Assessment Rubrics: simpler evaluation frameworks used in A1.2 for need significance screening; MCDA provides more rigorous multi-dimensional assessment for A1.4+.
• Governance Decision Brief: documents MCDA results for I2 review; scorecard tables embedded in the brief.