DECISION 312 – Decision Models            Class 7 

Spring Term 2, 2009‐2010             

Identifying Important Uncertainties  
(Tornado Charts, SensIt) 
 

1 
Today we will describe a tool for identifying the "most 
important" uncertainties in a problem. 
 
 

Initial Deterministic Probabilistic Decision

Structure Appraisal
Situation Analysis Analysis

Iteration

Decision Deterministic Decision Value of

Hierarchy Model Tree Information
A B C
1
2
3
4
5

Influence Deterministic Probability Decision

Diagram Sensitivity Distributions Quality

 
Source: Strategic Decisions 
Monte Carlo   Group.  Used with permission 
  Simulation 

• "Tornado charts" are used in decision tree and Monte Carlo analysis (as 
well as other contexts) to focus attention and effort on the key 
uncertainties. 
  2
Example:  Introducing Gatorade in China 
 
 
 
• Gatorade was Quaker Oats' largest brand worldwide; acquired by Pepsi in 
2000. 
• Gatorade has a strong presence in the U.S. and Canada and a growing 
presence in other parts of the world: Latin America, Europe, and now Asia. 
• Four flavors of Gatorade (Lemon, Orange, Peach, and Grapefruit) are sold 
in glass bottles and cans in China. 
• “Some estimates show as many as 300 million of the 1.3 billion people 
here could afford the likes of Crest toothpaste and Gatorade.” WSJ 
1/3/2000. But the number could be much lower. 
• Coke costs a quarter:  Can they charge 50 cents for Gatorade? 

  3
An Economic Evaluation Model: 
Financial Assumptions:  
Average price/unit $0.50
Inflation rate 5.00% per year
Per unit materials $0.15
Labor FTEs per million units 5
Labor rate $8,000 per year
Shanghai facility rental $1,000,000 per year
Marketing--Variable 20% of revenue
Marketing--Fixed $850,000 per year
SG&A Variable 20% of revenue
SG&A Fixed $5,000,000 per year  
Discount rate 10% per year  
 
Gatorade in China Projections
All amounts in 000s
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

Inflation factor 1.00 1.05 1.10 1.16 1.22 1.28 1.34 1.41 1.48 1.55

Unit sales 26,737 68,911 120,402 178,939 240,288 299,290 351,481 394,303 427,298 451,503
Revenue $13,368 $36,178 $66,372 $103,572 $146,036 $190,989 $235,509 $277,412 $315,657 $350,214

Cost of goods
Ingredients/ Bottling materials $4,011 $10,853 $19,911 $31,072 $43,811 $57,297 $70,653 $83,224 $94,697 $105,064
Labor $1,069 $2,894 $5,310 $8,286 $11,683 $15,279 $18,841 $22,193 $25,253 $28,017
Shanghai facility rental $1,000 $1,050 $1,103 $1,158 $1,216 $1,276 $1,340 $1,407 $1,477 $1,551

Gross profit $7,288 $21,380 $40,048 $63,057 $89,327 $117,137 $144,676 $170,588 $194,230 $215,582

Additional operating expenses

Marketing
Fixed $850 $893 $937 $984 $1,033 $1,085 $1,139 $1,196 $1,256 $1,319
Variable $2,674 $7,236 $13,274 $20,714 $29,207 $38,198 $47,102 $55,482 $63,131 $70,043
SG&A
Fixed $5,000 $5,250 $5,513 $5,788 $6,078 $6,381 $6,700 $7,036 $7,387 $7,757
Variable $2,674 $7,236 $13,274 $20,714 $29,207 $38,198 $47,102 $55,482 $63,131 $70,043

Net income (loss) -$3,909 $767 $7,050 $14,856 $23,802 $33,275 $42,632 $51,392 $59,324 $66,421

NPV $142,704  
  4
The sales forecast is based on a standard new product diffusion 
model, a variation on the Bass model.1 
• Customers try the product and some of them become repeat customers; the base 
case assumes 7% of "triers" become repeat customers. 
o Trial purchases assumed to be of a single unit. 
o Repeat purchasers are assumed to purchase 6 units per month. 
• Potential market size = 100,000,000 people 
• The fraction of the market who try the product for the first time each month is a 
linear function of the number of people who have tried the product: 
4.0%

3.5%
Proportion of population

3.0%
who tries in month

2.5%

2.0%
Slope = "Coef f icient of imitation" = 3%
1.5%

1.0%

0.5%
Intercept = "Coef f icient of innovation" = 0.5%
0.0%
0% 20% 40% 60% 80% 100%
Proportion of population that has alre ady trie d

 
                                                                 
1
 Bass, F.M., "A new product growth model for consumer durables," Management Science (1969). 
  5
The diffusion model in equations: 
 
 
  Base Assumption  Description
m  100 million  Market size:  the number of people that might try the product.
Coefficient of innovation: describes how quickly innovators will try 
a  .5% per month 
a new product.   
Coefficient of imitation: describes the strength of the word‐of‐
b  3% per month 
mouth effect in creating new triers.  
Fraction of trial purchasers in month k‐1 who convert to repeat 
c  7% 
purchasers in month k. 
Average units per trial: the number of units a trial purchaser will 
t  1 
buy in a month. 
Average units per repeat purchase: the average number of units a 
r  6 
repeat purchaser will buy in a month. 
 
⎛ Y (k − 1) ⎞ Q1 (k) = tT (k)
T (k) = ⎜ a + b ⎟(m − Y (k − 1))  
⎝ m ⎠    
Q2 (k) = rcY (k − 1)  
 
where 
T(k) = number of trial purchasers in month k. 
Y(k) = cumulative number of trial purchasers up to month k. 
Q1(k) = sales to trial purchasers in month k. 
Q2(k) = sales to repeat purchasers in month k. 
  6
Results from the diffusion model: 
 
New product diffusion model
All amounts in 000s

Cumulative Cumulative Unit sales to Unit sales to Total unit

Number of trial trial repeat trial repeat sales in Annual unit
Month purchasers purchasers purchasers purchasers purchasers month sales
0 800 800
1 520 1,320 56 520 336 856
2 532 1,852 92 532 554 1,087
3 545 2,398 130 545 778 1,323
4 558 2,956 168 558 1,007 1,565
5 571 3,527 207 571 1,241 1,813
6 584 4,111 247 584 1,481 2,066
7 598 4,709 288 598 1,727 2,325
8 611 5,320 330 611 1,978 2,589
9 625 5,945 372 625 2,235 2,859
10 638 6,583 416 638 2,497 3,135
11 652 7,234 461 652 2,765 3,416
12 665 7,900 506 665 3,038 3,704 26,737
13 679 8,578 553 679 3,318 3,997
14 692 9,271 600 692 3,603 4,295
15 706 9,977 649 706 3,894 4,600  
 
 

Unit sales (000) Customers (000)

100,000
45,000
40,000 Cumulative trial purchasers

35,000 75,000
30,000
25,000 50,000
20,000
15,000 25,000
Cumulative repeat purchasers
10,000
5,000
0
0
0 12 24 36 48 60 72 84 96 108 120
0 12 24 36 48 60 72 84 96 108 120
month M onthly trial purchasers month
                    
 

  7
Tornado Analysis: First, we assess base, low, and high values for 
each uncertain input. 
• Assess: 
o Low values are 10th percentiles: there is a 1‐in‐10 chance that the true 
value is below this amount. 
o Base case values are 50th percentiles or medians: there is a 50‐50 
chance that the true value will be above or below this amount. 
o High values are 90th percentiles: there is a 1‐in‐10 chance that the true 
value is above this amount. 
 
• Consistency in the definition of the scenarios is required for comparability:   
o Most people use 10th, 50th, and 90th percentiles.  
o Some suggest using 5th, 50th, and 95th percentiles instead of 10‐50‐90s. 

  8
Rough estimates will do – we will refine the ranges for the 
important uncertainties later.  
• Ranges for the example: 
 
 
 
Ranges
Input cell 10th %ile 50th %ile 90th %ile
Average price/unit $0.50 $0.30 $0.50 $0.60
Inflation rate 5.00% per year 0.00% 5.00% 15.00%
Per unit materials $0.15 $0.13 $0.15 $0.17
Labor FTEs per million units 5 3 5 7
Labor rate $8,000 per year $7,000 $8,000 $9,000
Shanghai facility rental $1,000,000 per year $900,000 $1,000,000 $1,400,000
Marketing--Variable 20% of revenue 12% 20% 25%
Marketing--Fixed $850,000 per year $750,000 $850,000 $1,000,000
SG&A Variable 20% of revenue 12% 20% 25%
SG&A Fixed $5,000,000 per year $3,500,000 $5,000,000 $6,500,000
Total potential customers 100,000,000 50,000,000 100,000,000 350,000,000
Coefficient of innovation 0.50% per month 0.01% 0.50% 1.00%
Coefficient of imitation 3.00% per month 1.00% 3.00% 10.00%
Percent trial purchasers => rep 7% 3% 7% 20%
Average units per trial purchase 1 1 1 3
Average units per repeat purch 6 3 6 25
Existing trials purchasers 800,000 650,000 800,000 1,000,000  

  9
Next we vary each input, one at a time, to see how changes 
affect the bottom line. 
• Set all non‐changing variables at their base value. 
• Results summarized in a "tornado" chart: 
SensIt 1.31 Academic

Average units per repeat purchase 3 25

Total potential customers 50,000,000 350,000,000

Percent trial purchasers => repeaters 3% 20%

Average price/unit $0.30 $0.60

Coefficient of innovation 0.01% 1.00%

Coefficient of imitation 1.00% 10.00%

Inflation rate 0.00% 15.00%

Marketing--Variable 25% 12%

SG&A Variable 25% 12%

Per unit materials $0.17 $0.13

Labor FTEs per million units 7 3

SG&A Fixed $6,500,000 $3,500,000

Labor rate $9,000 $7,000

Average units per trial purchase 13

Shanghai facility rental $1,400,000 $900,000

Existing trials purchasers 650,000 1,000,000

Marketing--Fixed $1,000,000 $750,000

($200,000) $0 $200,000 $400,000 $600,000 $800,000

NPV
 
  10
Building Tornado charts using SensIt: 
• SensIt is an Excel add‐in; available from the course website.  
• SensIt adds commands to the Add‐ins toolbar/ribbon. 
 

 
 
• Caveats: Cell ranges must be contiguous. 
    Output cells must be on the same worksheet as the input cells 
    No apostrophes in file or worksheet names. 
  11
SensIt Output ‐‐ Table: 
 
 
 
 
NPV
Corresponding Input Value Output Value Percent
Input Variable Low Output Base Case High Output Low Base High Swing Swing^2
Average units per repeat purchase 3 6 25 $49,618 $142,704 $732,254 $682,636 38.0%
Total potential customers 50,000,000 100,000,000 350,000,000 $48,272 $142,704 $614,452 $566,179 26.1%
Percent trial purchasers => repeaters 3% 7% 20% $36,320 $142,704 $488,455 $452,135 16.7%
Average price/unit $0.30 $0.50 $0.60 ($68,569) $142,704 $248,341 $316,910 8.2%
Coefficient of innovation 0.01% 0.50% 1.00% ($16,568) $142,704 $198,596 $215,164 3.8%
Coefficient of imitation 1.00% 3.00% 10.00% $79,387 $142,704 $250,733 $171,346 2.4%
Inflation rate 0.00% 5.00% 15.00% $104,394 $142,704 $260,936 $156,542 2.0%
Marketing--Variable 25% 20% 12% $98,689 $142,704 $213,129 $114,440 1.1%
SG&A Variable 25% 20% 12% $98,689 $142,704 $213,129 $114,440 1.1%
Per unit materials $0.17 $0.15 $0.13 $107,492 $142,704 $177,917 $70,424 0.4%
Labor FTEs per million units 7 5 3 $114,535 $142,704 $170,874 $56,340 0.3%
SG&A Fixed $6,500,000 $5,000,000 $3,500,000 $131,545 $142,704 $153,864 $22,319 0.0%
Labor rate $9,000 $8,000 $7,000 $133,901 $142,704 $151,507 $17,606 0.0%
Average units per trial purchase 1 1 3 $142,704 $142,704 $157,692 $14,987 0.0%
Shanghai facility rental $1,400,000 $1,000,000 $900,000 $139,729 $142,704 $143,448 $3,720 0.0%
Existing trials purchasers 650,000 800,000 1,000,000 $141,783 $142,704 $143,923 $2,140 0.0%
Marketing--Fixed $1,000,000 $850,000 $750,000 $141,588 $142,704 $143,448 $1,860 0.0%  
 

  12
SensIt Output – Tornado Chart: 

  13
The impact of uncertainty about a variable is measured by its 
"swing."  
 
• Swing   = (Value in high case) – (Value in Low Case) 
 
    = Width of the bar in the Tornado chart 
 
• Variables are sorted by their swing in the Tornado chart 
 
• Qualitatively, swing is determined by two things: 
o How uncertain we are about the variable: 
No uncertainty ⇒ No swing 
o How sensitive the value measure (e.g., NPV) is to changes in the 
variable: 
No sensitivity ⇒ No swing 

  14
Often most of the uncertainty in value is due to uncertainty in a 
small set of variables. 
• The "percent of variance explained" is a rough measure of the impact of 
one variable on the uncertainty in value:
o Variance for a variable is roughly proportional the square of the swing.  
(Exact if all variables have a normal distribution.) 
o Total variance is roughly equal to the sum of variances due to the 
individual variables.  (Assumes additivity and independence.) 
 
(Swing for this var)2
"Percent variance" for variable = (Total of Squared Swings) 
 
• Impact on uncertainty in value is proportional to the square of the swing.   
o Little is lost in ignoring uncertainty in variables with small swings. 
o May be able to capture 95% of variance with just a few variables. 

  15
What if two or more variables are highly correlated? 
• Worry about it:  If two or more variables are highly correlated, you should 
vary them simultaneously rather than independently: 
o Define scenarios that jointly specify high, medium, and low values for 
each variable and do sensitivity over the scenarios. 
o Define a multiplier that changes all of the dependent variables 
simultaneously. 
 
• Don't worry about it:  If one variable has little impact on its own, it is not 
likely to have much of an impact when moved with others. 
o Correlation with another variable can increase a variable's impact, but 
the increase will (typically) be less than doubling the "percent 
variance" associated with the variable.  
 
• Worry later:  If both variables are important, we will model the uncertainty 
in more detail later and can model the dependence then. 
  16
SensIt's – Spider Output (Table): 
 
NPV
Corresponding Input Value Input Value as % of Base Output Value
Input Variable Low Output Base Case High Output Low % Base % High % Low Base High Swing
Average units per repeat purchase 3 6 25 50.0% 100.0% 416.7% $49,618 $142,704 $732,254 $682,636
Total potential customers 50,000,000 100,000,000 350,000,000 50.0% 100.0% 350.0% $48,272 $142,704 $614,452 $566,179
Percent trial purchasers => repeaters 3% 7% 20% 42.9% 100.0% 285.7% $36,320 $142,704 $488,455 $452,135
Average price/unit $0.30 $0.50 $0.60 60.0% 100.0% 120.0% ($68,569) $142,704 $248,341 $316,910
Coefficient of innovation 0.01% 0.50% 1.00% 2.0% 100.0% 200.0% ($16,568) $142,704 $198,596 $215,164
Coefficient of imitation 1.00% 3.00% 10.00% 33.3% 100.0% 333.3% $79,387 $142,704 $250,733 $171,346
Inflation rate 0.00% 5.00% 15.00% 0.0% 100.0% 300.0% $104,394 $142,704 $260,936 $156,542
Marketing--Variable 25% 20% 12% 125.0% 100.0% 60.0% $98,689 $142,704 $213,129 $114,440
SG&A Variable 25% 20% 12% 125.0% 100.0% 60.0% $98,689 $142,704 $213,129 $114,440
Per unit materials $0.17 $0.15 $0.13 113.3% 100.0% 86.7% $107,492 $142,704 $177,917 $70,424
Labor FTEs per million units 7 5 3 140.0% 100.0% 60.0% $114,535 $142,704 $170,874 $56,340
SG&A Fixed $6,500,000 $5,000,000 $3,500,000 130.0% 100.0% 70.0% $131,545 $142,704 $153,864 $22,319
Labor rate $9,000 $8,000 $7,000 112.5% 100.0% 87.5% $133,901 $142,704 $151,507 $17,606
Average units per trial purchase 1 1 3 100.0% 100.0% 300.0% $142,704 $142,704 $157,692 $14,987
Shanghai facility rental $1,400,000 $1,000,000 $900,000 140.0% 100.0% 90.0% $139,729 $142,704 $143,448 $3,720
Existing trials purchasers 650,000 800,000 1,000,000 81.3% 100.0% 125.0% $141,783 $142,704 $143,923 $2,140
Marketing--Fixed $1,000,000 $850,000 $750,000 117.6% 100.0% 88.2% $141,588 $142,704 $143,448 $1,860  

 
• Note:  Spider plots do not include variables whose base case value is 0. 

  17
SensIt's – Spider Output (Plot): 
 
SensIt 1.42 Academic Version
$900,000 

$800,000 

Average units per repeat purchase
$700,000 
Total potential customers
Percent trial purchasers => repeaters
$600,000 
Average price/unit

$500,000  Coefficient of innovation
Coefficient of imitation
$400,000  Inflation rate
NPV

Marketing‐‐Variable
$300,000 
SG&A Variable
Per unit materials
$200,000 
Labor FTEs per million units

$100,000  SG&A Fixed
Labor rate
$0  Average units per trial purchase
Shanghai facility rental
($100,000) Existing trials purchasers
Marketing‐‐Fixed
($200,000)
‐50.0% 0.0% 50.0% 100.0% 150.0% 200.0% 250.0% 300.0% 350.0% 400.0% 450.0% 500.0%

Input Value as % of Base Case
 

  18
Example:  Risk analysis for a large offshore oil project 
• Study done in 1995.   
• Significant oil reserves had been   
discovered, but uncertainty 
about reserves remained. 
• Development requires building a 
production platform, drilling 
many wells, and building a 
transportation system.   
• The owner of the field was 
looking for a partner to share 
expenses and revenues.   
• There were complex tax and 
royalty considerations.   
• The purpose of the study was to 
value and look at the risks 
associated with the investment. 
  19
 

Uncertainty about production was described by low, medium, 
and high scenarios with different amounts of reserves. 
 
 
Year 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Years from Start 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Low Case (case 1)

Years to Production 7
Wells Drilled 6 4 8 7 4 9 3 2 2 2
Cumulative Well Count 6 10 18 25 29 38 41 43 45 47 47 47 41
Light Oil Production 16.7 27.7 34.3 34.7 28.8 22.3 17.3 13.1 9.7 7.4 5.0
Heavy Oil Production
Development Well Costs 46.4 73.6 138.7 128.3 73.6 155.3 52.7 35.8 35.8 33.8
Exploration Well Costs 18.0
Facilities Capex 5.0 13.0 15.0 81.0 144.0 288.0 550.0 45.0 96.0
Tankers Capex* 67.3 64.2 166.2 30.6

Mid Case (case 2)

Years to Production 7
Wells Drilled 6 4 8 7 4 4 2 3 2 2
Cumulative Well Count 6 10 18 25 29 33 35 38 40 42 42 42 36 32 24 17
Light Oil Production 25.2 36.1 36.1 36.1 36.1 34.3 29.6 25.6 22.3 19.3 16.8 14.2 12.0 10.2
Heavy Oil Production
Development Well Costs 46.4 73.6 138.7 128.3 73.6 41.2 37.8 58.3 35.8 33.8
Exploration Well Costs 36.0
Facilities Capex 5.0 13.0 15.0 81.0 144.0 288.0 550.0 45.0 96.0 38.0 36.0
Tankers Capex 67.3 64.2 166.2 30.6

High Case (case 3)

Years to Production 7
Wells Drilled 6 4 8 7 4 4 2 2 2 2
Cumulative Well Count 6 10 18 25 29 33 35 37 39 41 41 41 41 41 41 41
Light Oil Production 34.4 36.0 36.0 36.0 36.0 36.0 36.0 36.0 34.0 31.1 28.4 25.0 23.7 21.7
Heavy Oil Production
Development Well Costs 46.4 73.6 138.7 128.3 73.6 59.2 37.8 18.8 35.8 33.8
Exploration Well Costs 18.0 17.0
Facilities Capex 5.0 13.0 15.0 81.0 144.0 288.0 550.0 45.0 96.0 38.0 36.0
Tankers Capex 67.3 64.2 165.2 30.6  

20 
Uncertainty about future oil prices was similarly described using 
low, medium and high scenarios. 
 
 

35 Oil Price Scenarios

30
High Scenario
Oil Price ($/bbl WTI)

25 Mid Scenario

20
Low Scenario

15

10

5
1995 1997 1999 2001 2003 2005 2007 2009 2011 2013 2015
Year
 
 

• These corporate price forecasts (for WTI) had to be adjusted for the quality 
and location of the oil produced as well as exchange rates. 

  21
Modeling cost uncertainties: 
• Engineers had prepared detailed cost estimates for drilling, facilities, and 
other costs for each year, in each production scenario. 
 
• Uncertainty about these costs was captured by "multipliers." 
 
Example:  Facilities construction costs: 
o High scenario: All costs multiplied by 1.30 (increase by 30%) 
o Mid scenario:  All costs multiplied by 1.00. 
o Low scenario: All costs multiplied by .85 
Ranges provided by executive team. 
 
• There was more uncertainty about drilling costs, because of the new 
technologies being used and the potentially difficult conditions 
o Multiplier range:  Low = .80; Mid = 1.0; High = 1.5 
  22
The tornado analysis helped focus the analysis on the "most 
important" uncertainties. 
 
 
SensIt - Sensitivity Analysis - Tornado

Production Scenario 1 3

Oil Price Scenario 1 3

Exchange Rate 0.9 0.65

Brent/WTI Basis Ratio 86.0% 100.0%

Platform Cost Multiplier 1.30 0.85

Modeled as 
Drilling Cost Multiplier 1.50 0.80
uncertain 
Operating Cost Scenario 3 1

Resource Allowance Rate 0.0% 25.0%

Allowed Return on Investment 10.0% 15.0% Treated as 

Tanker CAPEX Multiplier 1.25 0.85
constants 

Number of Tankers 1 2

Tanker Fixed Costs 12.00 8.00

Variable Transshipment Costs 0.65 0.50

Tanker Port Charges 0.07 0.05

100.000 300.000 500.000 700.000 900.000 1100.000

NPV (@10%)

  23
Tornado charts in Crystal Ball: 
 
• CB has a tool for creating Tornado charts. 
o Must define distributions ("assumptions") before doing Tornado 
analysis in Crystal Ball. 
o You can however define "dummy" distributions to match specified 
percentiles and then use CB's tornado tool. 
o See CB documentation. 
 
• CB can also generate "sensitivity reports" that look a bit like Tornado 
charts but are quite different. 
o Report correlations between input uncertainties ("assumptions") and 
outputs ("forecasts") after running a simulation. 
o Bars are rank correlation coefficients or contribution to variance. 
o Not recommended. 
  24
Summary 
 
• Tornado charts help identify the most important uncertainties in a 
problem. 
o Build a deterministic model of the decision problem (i.e., identify the 
relationships among the elements), and  
o See how changes in the input uncertainties impact the bottom line. 
 
• The tornado chart helps you: 
o Understand the workings of the model: 
- Be sure you can explain the results of the tornado chart! 
- Helpful in debugging! 
o Decide which variables you should focus your attention on (i.e., assess 
probabilities from experts, gather more data, ...) 
 

  25