1 Managing Risk in an Uncertain World & The Danger of “The Number”June 2012 Adapted from the Palisade Corporation presentation, “The Power of Probabilities”, 2011

2 The World is Uncertain On one hand, it is no surprise that the world is uncertain Most things that happen in the news are “unexpected” Emergencies and disasters compound the natural uncertainties of the world On the other, we are driven by a need for “the number”, also known as: The “Planning Factor” The “Limiting Factor” The “Average”

3 Examples of UncertaintyThe number of patients that will need to be evacuated on any given day dependent upon a highly variable threat The number of people that will evacuate to Shreveport and subsequently need hospital admissions The number of empty, staffed beds in a hospital on any given day The number of patients that are on ventilators on any given day The number of CCATT teams that may be available for deployment on any given day

4 “The Number” is Dangerous“The Number,” once written, becomes set in stone “The Number” is disseminated to everyone involved “The Number’s” underlying assumptions – and errors - are forgotten “The Number” become the basis for big decisions

5 Books Have Been Written About It“A bigger problem with the deterministic approach is that once a value is generated, put down on paper, and incorporated into the business plan, it becomes gospel. Nobody questions it anymore; it’s in the plan so it must be true. The deterministic approach leads to complacency. We actually begin to believe the numbers we’ve generated…. This is dangerous.”

6 It’s All in the Problem DefinitionContext: A pandemic is occurring and it is estimated that 600 people will die if nothing is done Scenario 1 – which would you choose? Option A: 200 people will definitely be saved Option B: There is a 33% chance that 600 people will be saved, but a 67% chance that no one will be saved Scenario 2 – which would you choose? Option C: 400 people will definitely die Option D: There is a 33% chance that no one will die, and a 67% chance that 600 people will die A study by Kahneman and Tversky presented this problem to a group: In Scenario 1, 72% of people chose Option A, and 28% chose Option B In Scenario 2, 78% of people chose Option D, and only 22% chose Option C. Options A and C are identical and Options B and D are identical. In both A and C, 200 people live and 400 people die In both B and D, you have a 33% chance of saving all 600 and a 67% chance of losing all 600. People are not risk-averse, they are “loss-averse”. Given the chance for a sure-fire win, people will choose it no matter the lower probability of larger gains. However, when given a choice between a sure-fire loss and a chance of gains, people will opt for chance of gains. Hint: Options A and C are identical, as are options B and D

7 It’s All in the Problem DefinitionOur MIEP problem is the same Scenario 1 – Option A: X number of patients will definitely be evacuated Option B: There is a 33% chance that all patients will be evacuated, but a 67% chance that no one will be evacuated Scenario 2 – which would you choose? Option C: X number of patients will definitely not be evacuated Option D: There is a 33% chance that everyone will be evacuated, and a 67% chance that no one will be evacuated The differentiator is the willingness to gamble on the availability of resources not formerly included in TRANSCOMM’s planning model.

8 Compensating for UncertaintyWe employ a number of comfortable techniques to both deal with the uncertainty as well as to get “the number” We use 3-point estimates (best case, most likely case, and worse case) We use scoring methods (fit things into categories and scales) We use spreadsheets to run “what ifs” (we change one number at a time) We go with our “gut”

9 1: Dangers of the “3 Point Estimate”3 Point estimates Only give 3 possible outcomes of thousands Gives no probability to assess importance Provides no actionable direction Low and high estimates are individually thought to be very unlikely The sum (or product) of the extreme cases is even more unlikely The result is nearly useless, but gives you the most likely “number”

10 2: Dangers of “Scoring Methods”Like the 3-point estimate, Scoring Methods: Rely on subjective, qualitative judgment and perception Different qualitative descriptions of likelihood are different languages to different people Agreeing on scores or categories which have different underlying meanings to different people creates “illusion of communication” Arbitrary scales (like 1-5) compress wide range of values into artificially small, inaccurate “space” These scales imply that the intervals are regular No consideration given to dependency of factors on each other (correlation)

11 3: Dangers of “What Ifs” Correlations again: Things could be dependent on each other You get lost in the numbers Generates arguments about which assumptions are “correct” Gives arbitrary results Running lots of subjective “What-ifs” forces managers to pick “most likely” point assumptions to get “The Number”

12 4: Dangers of the “Gut” “Gut feel” assumptions taken as certainErroneous point estimates compounded in even the simplest situations Can’t figure out the problem, because individual factors seem “right” Gut feel simply guesses “The Number” (and is at least faster than the other methods)

13 What’s the Answer? Probabilistic Analysis

14 So What Does “Probabilistic” Mean?Using a range of values and associated probabilities for each input parameter (usually given as a probability function), and generating a histogram of values for each output calculation (sometimes called an output probability distribution).

15 What Does That Mean? Probabilistic means “in ranges” ANDIt means thinking in two dimensions: Not just “what if” But also “how likely”

16 Why is Probabilistic Analysis Important?Because the world is uncertain The LESS we know, the GREATER our need for probability

17 But What Does That Mean? Uncertainty means “unknown”Whether or not it can be “figured out” Whether or not it will be known in the future Uncertain means it could be this, or this, or this, or… But usually you have some idea of what it’s between – you know a range

18 Thinking in Ranges Start thinking of what could happen (without getting paranoid!) Estimate actual probabilities Get used to: “I think there is a 70% chance of a more patients being admitted than the average.”

19 Example: Probabilistic vs. 3-point EstimatesIt means not just: best case, worst case, most likely case But instead you might see: 10% chance of achieving your best case, 5% chance of encountering the worst case, and a 30% chance of the most likely case

20 Probabilistic Analysis : The Antidote to “The Number”With Probabilistic Analysis, you can answer: “What is the probability of meeting our target?” “What are the chances of not meeting the requirement?” “Where’s the graph?” These are questions you must ask that are not possible with spreadsheets alone Monte Carlo simulation turns static spreadsheets into probabilistic ranges

21 What Does Probabilistic Analysis Tell You?What your best case, worst case, and most likely case will be; How likely these cases are to occur, and Everything in between! You avoid perils of all other methods: You avoid getting lost in numbers You capture correlations between related variables Everyone can understand the results: common language Results are based on data, not arbitrary perceptions

22 Benefits of Probabilistic AnalysisIdentify drivers of risk Sensitivity analysis ranks variables according to impact on your output Understand which factors are most important Target specific variables that have the biggest impact Don’t waste resources on low-impact events, or allocate too much to low-probability events Finally….make better decisions Make the best decision possible given the information at hand, based on a balance of objective analysis and expert judgment