AI & AT

Module III
Chapter 2
Statistical Reasoning
Basics of Probability
Theory
Probabilitytheory is a
mathematical approach for
processing uncertain information
And we talk about a sample
space
What is a sample space?
Sample Space
A sample space is the set of all possible
events
For example, when we deal with a dice when
you toss a die which has got six faces the
possible outcomes are 1 2 3 4 5 6 so that is a
set
It can be discrete or continuous
For example, when I think of the possible
temperatures that can be there tomorrow
It can vary in a continuous range
So this sample space can be considered as a
set
What is a Probability?
Probability is a real number P(Xi)
where Xi represents any of these
events
So we call these to be the events
and any of these events is
associated with a real number
which denotes the likelihood of
the event to occur
And as you know it is a non-
negative value between 0 and 1
What is meant by mutually
exclusive events?
Two events cannot take place
together
When I throw a coin it cannot be
head and tail, it will be either
head or tail
Experimental Probability
Experimental probability is based on the
frequency of events
If I carry out a fair experiment of throwing
the die or if I take a biased coin and throw it
ten times may be it will not be five times
head and five times tail
If it is biased then may be I will get seven
times head and three times tail
But with that experiment being done quite a
number of times we can come to a fair
degree of association of the frequency of
the events
Subjective Probability
Subjective probability is based on
the expert’s assessment of the
whole thing
Independent Events
When two events are said to be
independent of each other, it
means that the probability that
one event occurs in no way
affects the probability of the
other event occurring
Ex. Rolling Dice and Flipping a
Coin
Joint Probability
The joint probability of two
independent events A and B is
given by P probability of A and B
that is the number of
intersections A and B divided by
the total number of elements in
the set and that can be computed
as the probability of A times
probability of B.
P(A∩B) = n(A∩B) /n(S) = P(A)*P(B)
Probability – Basic Axioms
 A value between 0 and 1
P(E) denotes the probability of an
event
P(True) = 1, i.e. certain event
P(False) = 0, i.e. impossible
event
P(A) = 1 – P(¬A)
P(AνB) = P(A) + P(B) – P(AΛB)
Dependent Events
Dependent events mean they
affect in some way
Two events are dependent if the
outcome or occurrence of the
first affects the outcome or
occurrence of the second so that
the probability is changed
Now, if two events are dependent
then that is the fact that one
event has occurred will certainly
influence the occurrence of
another event
This is known as conditional
probability
That is, conditional probability of
an event A given that event B
has already occurred that is given
P(A/B) = P(A∩B)/P(B)
P(B/A) = P(A∩B)/P(A)
P(A/B) = P(B/A).P(A) Bayes’ Rule
P(B)
Bayes’ Rule or Law or
Theorem
forms a basis of probabilistic reasoning
presented in "An Essay towards solving
a Problem in the Doctrine of
Chances" by Thomas Bayes (1701 –
1761) an English statistician and
philosopher
Bayes never published what would
eventually become his most famous
accomplishment; his notes were edited
and published after his death
by Richard Price
Example
Statement:
Money cannot buy happiness
(Source: Harvard study showing that
only 10% of happy people are rich)

What  you really want to know is what

percent of rich people are happy?
This would give a better idea of
whether becoming rich might make
you happy. 
Bayes' Theorem tells you how to
calculate this
other, reversed statistic using
two additional pieces of
information: 

1. The percent of people overall

who are happy
2. The percent of people overall
who are rich
The key idea of Bayes' theorem
is reversing the statistic using
the overall rates
It says that the fraction of rich
people who are happy is the
fraction of happy people who are
rich, times the overall fraction
who are happy, divided by the
overall fraction who are rich
So if 
1. 40% of people are happy; and
2. 5% of people are rich
And if the Harvard study is
correct, then the fraction of rich
people who are happy is:
0.1×0.4/0.05 = 0.8
So a pretty strong majority (80%)
of rich people are happy
Let's say the population of the whole
world is 1000, just to keep it easy
Then Fact 1 tells us there are 400 happy
people, and the Harvard study tells us
that 10% of these people are rich
So there are 40 people who are both
rich and happy
According to Fact 2, there are 50 rich
people altogether, so the fraction of
them who are happy is 40/50, or 80%.