Sensitivity Analysis

Purpose

Systematically vary one input at a time in the ROI model (the referenced method) to determine which assumptions the go/no-go recommendation depends on most—revealing whether the decision is robust across plausible ranges or fragile, hinging on a single optimistic assumption. Sensitivity analysis answers the question decision-makers actually need answered: “What would have to go wrong to change this recommendation?”

In A3.4, sensitivity analysis serves as the final stress test before the evaluation report reaches A3.5. If the recommendation flips from “go” to “kill” when churn increases by 3 percentage points, the governance forum needs to know. If the recommendation holds even under worst-case assumptions on all dimensions, the decision is straightforward.

When to Use

Use sensitivity analysis when:

• ROI model (the referenced method) is complete and shows positive expected NPV—test whether that result is robust
• Key inputs are uncertain—pilot-validated inputs (retention, support cost) carry less uncertainty than assumed inputs (growth rate, CAC at scale)
• A3.5 governance forum will challenge assumptions —sensitivity analysis pre-empts “What if…?” questions with prepared answers
• Comparing prototypes with similar expected NPV but different risk profiles—sensitivity reveals which is more fragile

Do NOT use when:

• ROI model already shows negative expected NPV —the project fails under expected assumptions; sensitivity analysis on an already-negative case adds little
• All key inputs are pilot-validated with narrow uncertainty bands—sensitivity analysis is most valuable for uncertain inputs

Sample Size and Duration

Effort: 6–10 hours per prototype (Financial Analyst: 4–6 hours; Product Owner: 2–4 hours interpreting results)

Duration: 1–2 days within A3.4 timebox (follows ROI modelling)

Output: Tornado chart, break-even thresholds, two-variable matrix, synthesis narrative

Prerequisites

• Completed ROI model: (the referenced method) with expected-case NPV, IRR, and payback period
• Input classification: Each ROI model input tagged as “pilot-validated” (lower uncertainty) or “assumed” (higher uncertainty)
• Plausible ranges: For each input, define pessimistic and optimistic bounds (from market data, pilot variance, or expert judgement)
• Decision threshold: What NPV constitutes “go”? (Typically >0, but some organisations require NPV > minimum return)

Complete Procedure

Step~1: Select Variables to Test (1 hour)

Prioritise inputs by uncertainty and expected influence on NPV. Typically 5–8 variables:

• Monthly churn rate: Pilot-informed but uncertain at scale (e.g. 5%–12%)
• Customer growth rate: Assumed; depends on marketing effectiveness (e.g. 5%–25%/month)
• ARPU: Partially validated via willingness-to-pay signals (e.g. 10–20)
• CAC: Assumed; depends on channel efficiency (e.g. 30–80)
• A4 development cost: Estimated by architecture team (e.g. 500K–900K)
• Monthly operating cost: Partially pilot-validated (e.g. 80K–140K/month)

Step~2: One-at-a-Time Analysis (2–3 hours)

For each variable, hold all others at expected values and vary the target across its plausible range. Record the NPV at each point.

Example (C001 Smart Checkout, expected NPV +$1.4M):

Reading the table: Growth rate and monthly churn produce the largest NPV swings (3.3M and 2.9M respectively). These are the killer variables—the assumptions the recommendation depends on most.

This data is typically visualised as a tornado chart: horizontal bars showing each variable's NPV range, sorted by swing magnitude (largest at top).

Step~3: Break-Even Analysis (1–2 hours)

For each killer variable, find the value where NPV = 0:

• Churn break-even: NPV = 0 at 11.5% monthly churn (expected: 8%; pilot: 8%). Interpretation: Churn must increase 44% from pilot levels to destroy value. Moderate buffer.
• Growth break-even: NPV = 0 at 4% monthly growth (expected: 15%). Interpretation: Growth must be 73% below expectations to destroy value. Strong buffer.
• ARPU break-even: NPV = 0 at 8.50 (expected: 15). Interpretation: ARPU must drop 43% from expected levels. Moderate buffer.

Decision insight: The recommendation is most sensitive to churn and ARPU—if both deteriorate simultaneously, NPV goes negative quickly. Growth rate alone is unlikely to kill the project.

Step~4: Two-Variable Sensitivity (optional, 1–2 hours)

For the two killer variables, create a matrix showing NPV under joint variation:

Reading the table: NPV is negative only in 3 of 9 cells—all involving high churn (12%) combined with low or moderate growth. The project is profitable in 6 of 9 scenarios. Key risk: if churn is worse than pilot levels and growth disappoints.

Step~5: Synthesise for A3.4 Report (1–2 hours)

Produce a sensitivity summary for the evaluation report:

1. Killer variables: Growth rate and churn rate dominate NPV sensitivity
2. Robustness assessment: Recommendation (Go) holds in 6 of 9 joint scenarios. Flips to negative only under simultaneous pessimistic churn and low growth.
3. Pilot evidence: Churn is pilot-validated (8%)—low probability of 12% unless product degrades at scale. Growth rate is assumed —highest-uncertainty variable.
4. A3.5 implication: Decision is robust. Monitor churn weekly in A4/A7. Growth rate uncertainty is acceptable given bounded downside (worst-case NPV -600K, vs. expected +1.4M).
5. Conditional recommendation: Include churn kill-trigger in A4 charter (e.g. if Month~3 churn >10%, trigger review).

Quality Criteria

1. Killer variables identified: Top 2–3 variables with largest NPV influence named
2. Both directions tested: Each variable tested at pessimistic and optimistic bounds
3. Break-even calculated: NPV = 0 threshold identified for each killer variable
4. Pilot vs. assumed distinguished: Variables classified by evidence quality (pilot-validated inputs have narrower uncertainty)
5. Joint sensitivity: At least one two-variable or “everything goes wrong” analysis
6. Decision-framed: Results expressed as “What would have to go wrong to change the recommendation?” not raw numbers
7. Actionable: Killer variables linked to A4 monitoring triggers and kill criteria

Theoretical Foundation

Seminal references:

• : Established sensitivity analysis (“what-if analysis”) as standard corporate finance practice: vary each input independently to measure its effect on NPV. Complemented by scenario analysis (vary multiple inputs simultaneously) and break-even analysis (find the input value where NPV = 0).
• : Demonstrated that decision-makers anchor on base cases and underweight tail risks. Sensitivity analysis forces explicit consideration of deviations from the expected case, counteracting anchoring bias.

Contemporary references:

• : Applied sensitivity analysis to stage-gate decisions: at each gate, identify the “killer variables”—the 2–3 inputs that determine whether the project creates or destroys value. If these variables are untested, the project needs more validation, not more financial modelling.

Types of Sensitivity Analysis

Challenges and Solutions

Challenge 1: Variable Selection Bias

Symptoms: Team only tests variables where they expect favourable results. “Let's check what happens if growth is even higher!” (upside sensitivity without downside).

Solutions: Test all variables in both directions. Mandate that every variable is tested at its pessimistic bound. If the team cannot define a pessimistic bound, they don't understand the variable well enough.

Challenge 2: Independence Assumption

Symptoms: OAT analysis shows each variable individually is safe. But in reality, high churn and high CAC and low growth occur together (correlated downside). OAT misses this.

Solutions: Complement OAT with two-variable sensitivity (Step~4) for the top 2–3 killer variables. For advanced analysis, use Monte Carlo simulation with correlated distributions. At minimum, run the “everything goes wrong” scenario: all variables at pessimistic bounds simultaneously.

Challenge 3: Paralysis by Sensitivity

Symptoms: Tornado chart shows large swings. Team concludes “too uncertain to decide.” A3.5 defers indefinitely.

Solutions: Uncertainty is expected at A3 stage. The question is not “Is there uncertainty?” but “Is the expected value positive with acceptable downside?” Frame for A3.5: “Under expected assumptions, NPV is +1.4M. Under simultaneous worst case, NPV is -600K. Probability of simultaneous worst case is <10%. Expected value is strongly positive.”

Relationship to Other Methods

Sensitivity Analysis receives input from:

• ROI Modelling (the referenced method): The base model that sensitivity analysis stress-tests
• Risk Assessment Matrix (the referenced method): High-rated risks identify which variables to prioritise
• Cohort Analysis (the referenced method): Pilot retention variance informs churn uncertainty range

Sensitivity Analysis provides input to:

• Three-Lens Evaluation (the referenced method): Viability lens robustness assessment
• Governance Forum (the referenced method): “What would have to go wrong?” framing for deliberation
• A4 Charter: Kill criteria and monitoring triggers derived from break-even thresholds

Tools and Templates

• Modelling: Excel Data Tables (one- and two-variable), Google Sheets (Goal Seek for break-even)
• Tornado charts: Excel (manual bar chart), Tableau, Python (matplotlib)
• Monte Carlo: @RISK (Palisade), Crystal Ball (Oracle), Python (NumPy/SciPy simulation)
• Templates: ROI model from the referenced method with sensitivity tab added

• D. Kahneman (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
• R. A. Brealey, S. C. Myers & F. Allen (2020). Principles of Corporate Finance. 13 ed. McGraw-Hill Education.
• R. G. Cooper (2017). Winning at New Products: Creating Value Through Innovation. 4 ed. Basic Books.