BAYESIAN ANALYSIS AND SOCIAL SCIENCES.

A TOOL FOR ALL TRADES

DR. STEFANIA PALADINI Coventry University

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

* In a very simple way, all B.A (=Bayesian Analysis) reduces to a single, but crucial, question: Do we change opinion when facts change?
And if so.... how do we modify our beliefs in the light of additional information?

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

Reverend Bayes was a Presbyterian minister and an amateur mathematician, living in England in the first part of XVIII century... ...and to be perfectly true, he is famous due to his friend Richard Price....

....who found, edited and published Bayes An Essay towards solving a Problem in the Doctrine of Chances in 1763

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

WHAT SO SPECIAL ABOUT BAYES (OR THE BAYES-PRICE FORMULA, BY OUR MODERN ACADEMIC STANDARDS)?

.... We need to remember that in 1700s probability was not as we know it today... it what essentially related to GAMBLING (if you remember its origin, that makes sense!)

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

Probability was until then related to answering questions such as:

What are my chances to deal four aces in three consequent runs of poker hands? This is something we call now the classical approach to probability

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

Bayes geniality was to pose, in OPPOSITE question:

a way, the

if somebody playing with me deals three consecutive 4 aces in a row, what is the probability is cheating (or the probability he/ she is just damn lucky)?

THIS IS CALLED THE INVERSE PROBABILITY PROBLEM AND IT CHANGED THE DISCIPLINE FOREVER

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

PROBABILIT(IES) HOW MANY OF THEM?

(Gonick L. & W. Smith, 1993)

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

Now what??

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

In the classical/frequentist approach, to calculate probability of a complex event we use something called conditional probability:

with

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

In the Bayesian approach, we use conditional probability in a slight different way:

H: is the hypothesis; E is the Evidence (your data) P(E given H) : it is the likelihood function P(H) is the prior P(H given E) is the posterior probability, ie, the resulting probability. Also called REVISED PROBABILIY

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

If you look at the formula, you will see that the posterior probability is proportional to two components: -The likelihood the distribution of the unobserved variable given data / ie. a probability model on the data observed - the Prior: our evaluation / subjective assessment

You see where the problem lies here.....

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

Lets do an example...
An initial suggestion though for people not used to probability calculations always better to visualise them, especially when dealing with conditional and inverse probability.

Thats why we use: Decision Trees

(I will show you both ways!)

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

The Ailing Plant Problem Before going on vacation for a week, you ask your spacey friend to water your ailing plant. Without water, the plant has a 90 percent chance of dying. Even with proper watering, it has a 20 percent chance of dying. And the probability that your friend will forget to water it is 30 percent.

(a) Whats the chance that your plant will survive the week? (b) If your friend forgot to water it, whats the chance itll be dead when you return? (c) If its dead when you return, whats the chance that your friend forgot to water it?

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

0.9 = die 0.3

0.9* 0.3 = 0.27

No water

0.9* 0.1 = 0.03 0.1= live =0.59 0.7* 0.8 = 0.56

0.8= live
0.7 water

0.7* 0.2 = 0.14

0.2=die 0.27/ (0.27+0.14) = 0.6585

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

(a) Whats the chance that your plant will survive the week? (b) If your friend forgot to water it, whats the chance itll be dead when you return? (c) If its dead when you return, whats the chance that your friend forgot to water it?

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

This means that if you plant is dead, your friend forgot about it at the 65.8%. In real life, you should find somebody more reliable!

However, this example shows that the initial assessment, ie, the chances YOU JUDGE your friend will forget to water it (30%) is decisive in obtaining a final result IT IS THE (IN)FAMOUS PRIOR (again)

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

EVERYTHING IS IN THE PRIOR -CHOOSE IT WISELY AND RECURSIVELY -ADJUST IT TO THE NEW FACTS!

Posterior becomes the new Prior in successive runs!

(Titterington, 1982)

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

BAYESIAN ANALYSIS A TOOL FOR ALL TRADES. Fields in which Bayesian analysis is particularly welcome and widely used: * Decision making * Quantitative Finance * Risk assessment / evaluation * Environmental analysis * Oil & Gas E&P .... and lets not forget health & natural science

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

BAYESIAN ANALYSIS AND SECURITY The use of Bayesian techniques (no matter the way they have been defined) in security is long established. A few examples: - Turings Machine and the Enigma code - The US Navy and the Soviet Submarine - The Rand Corporation and the assessment of the likelihood of a nuclear war

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

BAYESIAN ANALYSIS AND INTELLIGENCE CIAs interest as well is not so recent Zlotnicks article Bayesian Theorem for Intelligence analysis is from the 1970s.... He identifies three features which distinguish B.A from conventional intelligence analysis: * Quantify probabilistic judgements * In BA = using a set of all (alternative) hypotheses no cognitive bias * BA : focusing on single pieces of evidence in a systematic way instead than on a whole body this also is known to reduce bias

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

Which are the intelligence areas where BA looks more promising?

a few have been identified in time.

* Strategic warning
: weighting odds of two competitive H. (binomial) attack/ no attack from another state

* Terrorism
: the US Center for Risk and Economic Analysis of Terrorism Events (CREATE) routinely uses BA in its evaluations this is understandable, given the enhanced randomness of terrorism VS war

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

BAYESIAN ANALYSIS AND INTELLIGENCE

Recently (in the last 10 years) a series of tools, compounding BA with other quantitative methods, have been developed such as: Multi-Entity Bayesian networks (MEBNs) Hidden Markov Models (HMMs) Bayesian networks (BNs)

Generally for evaluating asymmetric threats

( Kardes & Hall, 2005)

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

STILL.... A lot of suspect and mistrust for the intelligence application of Bayesian techniques.

Why?
For a start: Bayesian reasoning is very counterintuitive

Bayesian math points to a fairly slow learning curve (Blair, 2004)

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

Intelligence Analysis and Biases why do we fail? In making rough probability judgments, people commonly depend upon one of several simplified rules of thumb that greatly ease the burden of decision. Using the "availability" rule, people judge the probability of an event by the ease with which they can imagine relevant instances of similar events or the number of such events that they can easily remember. With the "anchoring" strategy, people pick some natural starting point for a first approximation and then adjust this figure based on the results of additional information or analysis. Typically, they do not adjust the initial judgment enough. Expressions of probability, such as possible and probable, are a common source of ambiguity that make it easier for a reader to interpret a report as consistent with the reader's own preconceptions. The probability of a scenario is often miscalculated. Data on "prior probabilities" are commonly ignored unless they illuminate causal relationships. (Heier, 1999)

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

Some studies (Gigerenze & Hoffrage, 1995) have found out that human brain has difficulties figuring probabilities: - 1 out of 100 seems better than 1% (frequency instead of probability) - Even better - NATURAL FREQUENCES (ie, one in which the information about the prior probability is included in presenting the conditional probabilities; 80 items out of 100 has the character in exam)

This has been already discovered in intelligence studies applying Bayesian analysis

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

... On another hand ie, from the quantitative side of the spectrum - there are the century-old suspicions about the PRIOR!

How can we really trust a hunch??

In a way, it seems not to different than gambling on a hypothesis (again) This is the typical criticism by the frequentist approach

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

And finally....there are some common problems which affect ALL kind of intelligence analysis, and not just B.A. such as.... * misjudging evidence * confounding cause and effect * nonstationarity (short life span of the evidence collected) * non independences of the H evaluated BA is not a catch-all remedy but it is a powerful tool is used in the appropriate way!

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

(1) General References

(D.M.Titterington, 1982, Irreverent Bayes, BIAS 9 (1)-16-18) Gonick Ll. & W. Smith, 1993, The Cartoon Guide to Statistics, New York, NY : Collins A gentle introduction to Bayes: http://yudkowsky.net/rational/bayes Gigerenzer G.& U. Hoffrage 1995 How to Improve Bayesian Reasoning Without Instruction: Frequency Formats, Psychological Review, 102(4)684704 A nice (and useful) Bayesian applet: http://psych.fullerton.edu/mbirnbaum/bayes/BayesCalc.htm Iversen, Gudmund R. Bayesian Statistical Inference. Beverly Hills: Sage Publications, Inc., 1984. Steven Strogatz, Chances are http://opinionator.blogs.nytimes.com/2010/04/25/chances-are/

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

(2) References : Bayes & Intelligence

Zlotnick. J.(1970) Bayes' Theorem for Intelligence Analysis, available at https://www.cia.gov/library/center-for-the-study-of-intelligence/kentcsi/vol16no2/html/v16i2a03p_0001.htm Elisabeth Pat-Cornell (2001), Project: "Local risk assessment in a crisis risk management context: a tactical application of the Bayesian approach to intelligence analysis in a dynamic situation available at http://create.usc.edu/2011/03/bayesian_approach_to_intellige.html Pate-Cornell, M. E.(2002): "Fusion of Intelligence Information: A Bayesian Approach", Risk Analysis, Vol. 22, No. 3, 2002. Pp. 445-454. Hong Y. and G. Apostolakis (1992): "Conditional Influence Diagrams in Risk Management", Risk Analysis, Vol. 13, No.6, Pp. 625-636. Kardes, E., and Hall, R (2005): "Survey of Literature on Strategic Decision Making in the Presence of Adversaries", Report, National Center for Risk and Economic Analysis of Terrorism Events, University of Southern California, Los Angeles, CA. Hausken, K. (2002). Probabilistic risk analysis and game theory, Risk Analysis, Vol.22. Heuer, R. J. (1999) Psychology of Intelligence Analysis, CIA Blair, Bruce. The Logic of Intelligence Failure. Center for Defense Information, March 9, 2004, http://www.cdi.org/blair/logic.cfm Fisk, Charles E. The Sino-Soviet Border Dispute: A Comparison of the Conventional and Bayesian Methods for Intelligence Warning. Studies in Intelligence 16, no. 2 (1972): 52-62

BAYESIAN ANALYSIS AND SOCIAL SCIENCES

Sharon Bertsch McGrayne

THE THEORY THAT WOULD NOT DIE How Bayes Rule Cracked the Enigma Code, Hunted Down Russian Submarines and Emerged Triumphant From Two Centuries of Controversy
2011, Yale University Press.