Calibration

Good morning!

INTRODUCTION
Chad Weinman

INTRODUCTION
Chad Weinman

CALIBRATION
What, Why, How

WHAT?
a method for gauging and improving an individuals ability to make better estimates

WHY?
Heres a question to get us started: Who among us always gets complete & perfect quantitative data for their FAIR analyzes?

N obo dcomplete Who among us always gets y & perfect do quantitative data for their es! FAIR analyzes?
Heres a question to get this started:
8

WHY?

FAIR CONCEPTS
Probability vs. Possibility Objective vs. Subjective Accuracy vs. Precision

FAIR CONCEPTS
Probability vs. Possibility Objective vs. Subjective Accuracy vs. Precision

10

FAIR CONCEPTS
Probability vs. Possibility Objective vs. Subjective Accuracy vs. Precision

11

FAIR CONCEPTS
Probability vs. Possibility Objective vs. Subjective Accuracy vs. Precision

12

FAIR CONCEPTS
Probability vs. Possibility Objective vs. Subjective Accuracy vs. Precision

13

NOT ACCURATE

t N o d. oo G

NOR PRECISE
14

ACCURATE
ne fi s. m thi I h it w

NOT PRECISE
15

ACCURATE
is is ! h d T oo G

& PRECISE
16

NOT ACCURATE

t ! n s Do t h i e ik l

PRECISE
17

OKAY:
We now are all comfortable again with the difference between accuracy and precision. But how and where does this directly relate in our fair modeling?

18

LESSON:
As FAIR practitioners when we look to estimates and ranges, we should aim for accuracy with hint of precision. This is what decision-makers prefer. Its even what my wife prefers :)

19

FAIR
Measuring Risk (quantitative)

Loves Uncertainty

Utilizes Estimations & Ranges to Create Distributions

Min, M/L, Max values + Condence

20

FAIR
Measuring Risk (quantitative)

IS Loves Uncertainty E R E RE ON H H E TI W RA IB TS Utilizes Estimations & Ranges to Create Distributions AL FI C

Min, M/L, Max values + Condence

21

Example: Threat Event Frequency

A DISTRIBUTION
22

Example: Threat Event Frequency

Form the ends of our distribution

A DISTRIBUTION
23

Example: Threat Event Frequency

Shape the curve

A DISTRIBUTION
24

Example: Threat Event Frequency

12

18

24

30

THE DISTRIBUTION
25

Example: Threat Event Frequency

12

18

24

30

MONTE CARLO
26

Example: Threat Event Frequency

12

18

24

30

MONTE CARLO
27

Example: Threat Event Frequency

12

18

24

30

NOW WHAT IF WERE NOT CALIBRATED?

28

Example: Threat Event Frequency

!
0 6 12 18 24 30

HOLD UP - WHATS THAT?

29

Example: Threat Event Frequency

!
0 6 12 18 24 30

REAL TEF FOR CURRENT YEAR

30

Example: Threat Event Frequency

!
0 6 12 18 24 30

WHAT IF WERE CALIBRATED?

31

Example: Threat Event Frequency

!
0 6 12 18 24 30

WHAT IF WERE CALIBRATED?

32

Example: Threat Event Frequency

!
0 6 12 18 24 30

WHAT IF WERE CALIBRATED?

32

FOOD 4 THOUGHT
Research in Decision Physiology starting in 70-80s have extensively shown few people are naturally calibrated. But theres hope!

33

HOW
Calibration believe it or not is improved by practice. We learn whether we are by nature over-condent or under-condent.

34

ONE MORE THING:

* If your range is larger than someone elses, youre not wrong. * You are correct if you are 90% condent in each side of your range.

35

LETS GET STARTED...

36

ONE

WHAT IS THE AVG. WINGSPAN OF 747?

37

TWO IN 1938 A BRITISH STEAM LOCOMOTIVE SET A NEW SPEED RECORD BY GOING HOW FAST? (IN MPH)
38

THREE WHAT YEAR WAS FIRST SEASON IN THE MAJOR LEAGUE FOR THE CLEVELAND INDIANS?
39

FOUR

HOW MANY INCHES LONG IS A US $20 BILL?

40

FIVE ON AVERAGE, WHAT PERCENTAGE OF TOTAL SOFTWARE DEVELOPMENT PROJECT EFFORTS IS SPENT ON DESIGN?
41

SIX

HOW DEEP BENEATH THE SEA WAS THE TITANIC FOUND?

(IN MILES)

42

SEVEN WHAT IS THE AVG. TEMPERATURE IN OF THE ABYSSAL ZONE

(WHERE OCEANS ARE > 6,500 FEET DEEP)

43

EIGHT

HOW MANY YEARS WAS NELSON MANDELA IN PRISON?

44

NINE

HOW OLD WAS THE YOUNGEST PERSON TO FLY INTO SPACE?

45

TEN FINAL QUESTION WHAT IS THE ALTITUDE OF THE HIGHEST MOUNTAIN THAT I HAVE SUMMITED?
(IN FEET)

46

HOW DID WE DO?

47

?
Thank you! Chad Weinman chad@redpointrisk.com LinkedIn
48