UNIVERSITY OF DAR ES

SALAAM
MBEYA COLLEGE OF HEALTH
AND ALLIED SCIENCES
LECTURE NOTES ON
PROBABILITY THEORY
 Introduction
• Probability theory is concerned with the analysis of
phenomena that take place in in-deterministic, in other words
random circumstances.
• The theory of probability provides mathematical models for
real world phenomenon involving randomness.

• The concepts involved in probability theory are:

i. Random Experiment
ii. Sample Space
iii. Events
iv. Probability
 1. Random Experiment
• We are all familiar with the importance of experiments in
science.
• Experimentation is useful to us because we can assume that
if we perform certain experiments under very nearly
identical conditions, we will arrive at results that are
essentially the same.
• In these circumstances, we are able to control the value of
the variables that affect the outcome of the experiment.
• However, in some experiments, we are not able to ascertain
or control the value of certain variables so that the results
will vary from one performance of the experiment to the
next, even though most of the conditions are the same.
• These experiments are described as random experiments.
 Random Exp cont…
• Each performance in a random experiment is called a trial.
• The result of a trial in a random experiment is called an
outcome, an elementary event of a sample point.
• Outcomes of a trial are said to be equally likely if taking into
account all the relevant evidences, there is no reason to
expect on in preference of the other. We say that we have
equally likely events.
• Examples or random experiments are:-
i. Flipping a coin.
ii. Rolling a die.
iii. Sexual intercourse (copulation) for reproduction.
iv. Gambling.
v. Playing football.
vi. Surgical procedure
 2. Sample Space
• A set S that consists of all possible outcomes of a random
experiment is called a sample space, and each outcome is
called a sample point.

• The sample space resulting from flipping a coin is {H, T}.

If a coin is flipped twice or two coins are flipped, then the
sample space become {HH, HT, TH, TT}.
• The possible outcomes after rolling a die is {1, 2, 3, 4, 5, 6}
• The sample space resulting from playing a football is {Win,
Draw, Loose}.
• The possible outcome of gambling is {Win, Loose}.
• Possible outcome of surgical procedure is {Live, Die}.
 3. Events
• An event is a subset A of the sample space S, i.e., it is a set
of possible outcomes.
• Of all possible outcomes in the sample space of an
experiment, some of them satisfy a specified description,
and these are termed as events.
• Event A will occur or it will not occur during the course of
experiments.
• An event consisting of a single point of S is called a simple
or elementary event.
• As particular events, we have S itself, which is the sure or
certain event since an element of S must occur, and the
empty set ∅, which is called the impossible event because
an element of ∅ cannot occur.
 Events cont…
 By using set operations on events in S, we can obtain other
events in S. For example, if A and B are events, then
A ∪ B is the event “either A or B or both.” A ∪ B is called the
union of A and B.
 A ∩ B is the event “both A and B.” A ∩ B is called the
intersection of A and B.
A′ is the event “not A.” A′ is called the complement of A.
 A – B = A ∩ B′ is the event “A but not B.” In particular,
A′ = S – A
 If the sets corresponding to events A and B are disjoint, i.e.,
A ∩ B = ∅, we often say that the events are mutually exclusive.
 This means that they cannot both occur.
We say that a collection of events A1, A2, … , An is mutually
exclusive if every pair in the collection is mutually exclusive.
 Examples:
1. What is the intersection of A = { x is odd } and B = { x is
between -4 and 6 }, where the elements of the two sets are
integers?
Since "intersection" means "only things that are in both
sets", the intersection will be all the numbers which are
both odd and between –4 and 6. {–3, –1, 1, 3, 5}.
2. What is the union of A = { x is a natural number between 4 and
8 inclusive } and B = { x is a single-digit negative integer }?
Since "union" means "anything that is in either set", the
union will be everything from A plus everything in B. Since
A = { 4, 5, 6, 7, 8 } and B = { –9, –8, –7, –6, –5, –4, –3, –2, –1
}, then their union is: { –9, –8, –7, –6, –5, –4, –3, –2, –1, 4, 5,
6, 7, 8 }
 3. PROBABILITY
 The concept of probability provides us with the idea on how to
measure the chances of possible outcomes.
 In any random experiment there is always uncertainty as to
whether a particular event will or will not occur. The concept
of probability enables us to quantify these uncertainties.
 As a measure of the chance, or probability, with which we can
expect the event to occur, it is convenient to assign a non-
negative number between 0 and 1.
 If we are sure or certain that an event will occur, we say that
its probability is 1.
 If we are sure that the event will not occur, we say that its
probability is zero.
 Estimation of Probabilities
There are two important procedures by means of which we can
estimate the probability of an event.
1. Classical Approach:
 If a trial results into n exhaustive mutually exclusive and
equally likely events and m of them are favourable to the
happening of an event A, then the probability of happening
of event A is m/n.
2. Empirical or Frequency Approach:
 If a trial is repeated number of times under essentially
homogeneous and identical conditions, then the limiting value
of the ratio of the number of times the event A happens to the
number of trials as the number of trials becomes indefinitely
large is called the probability of happening of the event.i.e
lim m/n as n becomes large.
 Some Basic Properties of Probability
 For every event A in sample space S, 0 ≤ P(A) ≤ 1.
i.e a probability lies between 0 and 1.
 For any number of mutually exclusive events A1, A2, …, An
in S, P(A1 ∪ A2 ∪ … An ) = P(A1) + P(A2) + …+ P(An)
In particular, for two mutually exclusive events A1 and A2 ,
P(A1 ∪ A2 ) = P(A1) + P(A2)

 If probability of event A is 1, ‘A’ is called a certain or sure

event, and if probability of event A is zero, ‘A’ is called and
impossible event.
 If A′ is the complement of A, then P( A′ ) = 1 – P(A).

 For any events A and B, P(A) = P(A ∩ B) + P(A ∩ B′ )

 Properties cont…
 If A and B are any two events, then
P(A ∪ B) = P(A) + P(B) – P(A ∩ B)

More generally, if A1, A2, A3 are any three events, then

P(A1 ∪ A2 ∪ A3) = P(A1) + P(A2) + P(A3) – P(A1 ∩ A2) –
P(A2 ∩ A3) – P(A3 ∩ A1) + P(A1 ∩ A2 ∩ A3).

 If a sample space S consists of a finite number of outcomes A1,

A2, …, An, then
P(A1) + P(A2) + … + P(An) = 1.
 Conditional Probability
 Refer to the probability of an event given that the other event
has happened
 Let A and B are two events such that P(A) > 0
 The probability of B given that A has occurred, P(B | A) is
given as:
P ( B | A) = P( A ∩ B) / P( A)

 The probability of A given that B has occurred, P(A | B) is

given as:
P ( A | B) = P( A ∩ B) / P( B), provided that P(B)>0.

 Thus, the probability of the intersection of A and B is given as

P( A ∩ B) = P( A) P( B | A) or
P( A ∩ B) = P( B) P( A | B)
 Independent Events
 If P(B | A) = P(B), i.e., the probability of B occurring is not
affected by the occurrence or nonoccurrence of A, then we say
that A and B are independent events. This is equivalent to:
P( A ∩ B) = P( A) P( B)

 We say that three events A1, A2, A3 are mutually independent if

they are pairwise independent.
P(Aj ∩ Ak) = P(Aj)P(Ak) j ≠ k where j,k = 1,2,3
and
P( A1 ∩ A2 ∩ A3 ) = P( A1 ) P( A2 ) P( A3 )
Both of these properties must hold in order for the events to be
mutually independent.
Example 1

In a study area three insect species (A, B, C) can be found. We

place an insect trap, and after some time we look at the insects
that have been caught. Let

E1 : a specimen of species A can be found in the trap; P(E1) = 0.1;

E2 : a specimen of species B can be found in the trap; P(E2) = 0.2;

E3 : a specimen of species C can be found in the trap; P(E3) = 0.3:

Assuming that the events E1, E2, E3 are independent, find the
probability of the following events:
Example 1 cont…
a. (i) There is no specimen of species A in the trap.
(ii) There is no specimen of species B in the trap.
(iii) There is no specimen of species C in the trap.
b. There is an insect in the trap.
c. There is no insect in the trap.
d. There are specimens of all the three species in the trap.
e. There are specimens of at most two species in the trap.

Example 2
At a workplace the ratio of the women workers is 0.86. It is
known that 15% of the women working there suffer from
pollen allergy. What is the probability that a randomly chosen
worker is a women suffering from pollen allergy?
Example 3

Assume that boys and girls have the same chance of being
born and staying alive until a given age. Show that the
following two events relating to that age are independent in a
family with three children.
E := there are both boys and girls in the family;
F := there is at most one boy in the family

Example 4
At a job skill test 6 persons out of 100 suffer from physical
handicap and 4 from sensory handicap. Both handicaps were
detected in one person.
a) What is the probability that a randomly selected person
suffering from physical handicap suffers from sensory
handicap as well?
 b) What is the probability that a randomly chosen person who
does not suffer from sensory handicap does not suffer from
physical handicap?

Example 5
In a Drosophila population 40% of the individuals have a wing
mutation, 20% of them have an eye mutation, and 12% have
both wing and eye mutations.
a) What is the probability that a randomly chosen individual of
this population has at least one of the mutations?
b) What is the probability that a randomly chosen individual of
this population has exactly one of the mutations?
Example 6
Families with two children can be classified into four groups
concerning the sexes of the children if we take into account the
order of births. By denoting the birth of a boy by b, and the
birth of a girl by g, the sample space is S:={(b, b); (b, g); (g, b);
(g, g)}. Assume that the four events have the same chance of
occur-ring. What is the probability that in a randomly chosen
family with two children both children are boys provided that
there is a boy in the family?
 The Total Probability Theorem
• In applications it often happens that we cannot determine
directly the probability of an event, but we know its
conditional probabilities given other events.
• If all the events in a complete set of events have positive and
known probabilities, and we know the conditional probability
of an event given each event of the complete set.
• Then the total probability theorem allows us to calculate the
probability of the given event.
• If the events F1, F2,…, Fn with positive probabilities form a
complete set of events in a probability space, then the
probability of any event E can be given as
P(E) = P(E|F1) P(F1) + … + P(E|Fn) P(Fn)
• In a particular case of the theorem the complete set of events
consists of two elements (F and F’), Then:-
P(E) = P(E|F) P(F) + P(E|F’) P(F’)

Example 7
At a school a flu epidemic rages. The first-year students go to
three classes, 25 to class 1.A, 25 to class 1.B, 24 to class 1.C.
In class 1.A 40%, in class 1.B 56%, and in class 1.C 50% of
the pupils have got the flu. What is the probability that a
randomly chosen first-year student has got the flu?
 BAYES THEOREM
• In many cases it is not the probability of an event which we are
interested in, but the question of what role the occurrence of
another event has played in the occurrence of the given event.
• Bayesian statistics named after Rev. Thomas Bayes (1702-
1761) is used to answer such questions.
• Bayes Theorem for probability events A and B
p( B | A) p( A)
p( A | B) 
p( B)
Or for a set of mutually exclusive and exhaustive events
p ( B | Ai ) p( Ai )
p ( Ai | B) 
 j p( B | A j ) P( A j )
• P(A) is called “A prior Probability” because it exists before
we gain any information from the experiment itself.
• P(B|Ai) is called “Likelihoods” because it indicates how likely
the event A under the consideration is to occur given a prior
probability.
• P(Ai|B) is called “Posterior probability” because it is
determined after knowing the results of the experiment.

Example 8
• A new HIV test is claimed to have “95% sensitivity and 98%
specificity”
• In a population with an HIV prevalence of 1/1000, what is the
chance that a patient testing positive actually has HIV?
 Solution
Let A be the event patient is truly positive, A’ be the event that
they are truly negative
Let B be the event that they test positive
• We want p(A|B), given:-
• P(A)= 0.001
• “95% sensitivity” means that P(B|A) = 0.95
• “98% specificity” means that P(B|A’) = 0.02
So from Bayes Theorem
p( B | A) p( A)
p( A | B) 
p( B | A) p( A)  p( B | A' ) p( A' )
0.95  0.001
  0.045
0.95  0.001  0.02  0.999
Thus, only 4.5% of people testing positive will in fact have have
HIV.
• So the vital issue in this example is how should this test result
change our prior belief that the patient is HIV positive?
• The disease prevalence (p=0.001) can be thought of as a
‘prior’ probability.
• Observing a positive result causes us to modify this probability
to p=0.045 which is our ‘posterior’ probability that the patient
is HIV positive.
Example 9
A disease occurs in 0.5% of population. A diagnostic test gives
a positive result:-
– in 99% of people that have the disease.
– in 5% of people that do not have the disease (false
positive).
A random person from the street is found to be positive on this
test. What is the probability that they have the disease?
Example 10
In a town there is an equal number of men and women. From
100 men 5 are red-green colour deficient on an average, while
from 10, 000 women there are 25. We choose a red-green
colour deficient person at random. What is the probability that
the chosen person is a man?