Sugata Munshi

Electrical Engg Deptt, JU

MOTIVATION:
1. The term PROBABILITY refers to the study of randomness
and uncertainty.
2. In any situation in which one of a number of possible o/cs may
occur, the discipline of probability provides methods for
quantifying the chances, or likelihoods, associated with the
various o/cs.

3. The language of probability is constantly used in an informal

manner in both written & spoken contexts.

4. Examples include such statements as

It is likely that the paytm share value will increase by the end of
the year,
There is a 5050 chance that the incumbent will seek re-election,
There will probably be at least one section of that course offered
next year,
The odds favor a quick settlement of the strike,
and
It is expected that at least 20,000 concert tickets will be sold.

Certain, Impossible and Random Events:

Proposition 1: In every realization of a set of conditions , there occurs an

event A.
ExIf water at atm pressure (760 mm of Hg) is heated above 100C (the
set of condns ), it is transformed into steam (event A).
Or
For any chemical reaction of substances, w/o exchange with the
surrounding medium (the set of condns ), the total quantity of substance
(matter) remains unchanged (event A)law of conservation of matter.
 Sugata Munshi
Electrical Engg Deptt, JU
1. An event A that unavoidably occurs for every realization of a set of
conditions , is called certain (sure).
2. If the event A definitely cant occur for every realization of a set of
conditions , it is called impossible.
3. If the event A may or may not occur for every realization of a set of
conditions , it is called random.
Conclusion:
Any event can be considered as certain, impossible or random only with
respect to some definite set of conditions.

Mere assertion of the randomness of any event A is of very restricted

cognitive interest; it simply amounts to stating that the set of conditions
does not reflect the entire collection of reasons necessary and sufficient for
the occurrence of A.
Such an indication can only serve as a stimulus for determination of the
conditions of occurrence of A, but of itself does not yield us any +ve
knowledge.

There is a broad range of phenomena in which, given a repeated realization of a

set of condns , the portion of that range of cases when event A occurs only
occasionally deviates to any substantial degree from some average value, which
can thus serve as a characteristic indicator of the mass scale operation.

For such phenomena, not only can one say that event A is random but also a
quantitative estimate of the possibility of its occurrence.

Modeling random experiments:

A random experiment is any activity or process whose o/c is subject

to uncertainty.
Random experiment outcomes are non-deterministic never possible to
predict outcome accurately , in spite of prior knowledge of outcomes for
large no. of performances.
 Sugata Munshi
Electrical Engg Deptt, JU

Although the word experiment generally suggests a planned or

carefully controlled laboratory testing situation, we use it here in a
much wider sense.
Thus experiments that may be of interest include:

1. tossing a coin once or several times,

2. selecting a card or cards from a deck,

3. weighing a loaf of bread,

4. obtaining blood types from a group of individuals,

Inspection of a given item during production.

Observing the failure-free operation time of a system.
Observing the repairing time of a system.
As introduced in 1933 by A.N. Kolmogorov , mathematical
model of an experiment with random o/c is a triplet also
called probability space.

Sample Space

Field of Events .

Probability.
 Sugata Munshi
Electrical Engg Deptt, JU
Sample Space or Sample Description Space S set of all
possible o/cs of an expt. More precisely it is the set of all
possible distinct, indecomposable measurements that could be
observed.

Outcome = result of an expt. Possible Outcomes are

mutually exclusive in the sense that they cant occur
simultaneously.

1. In our study of probability, we will be interested not

only in the individual o/cs of S but also in various
collections of o/cs from S..
Simplest expts to which probability applies are those
with 2 possible o/cs.
One such expt consists of examining a single fuse to see
whether it is defective. The sample space for this expt can
be abbreviated as S = {N,D}
where N represents not defective, D represents defective,
& the braces are used to enclose the elements of a set.

Another such expt would involve tossing fair coin and

noting whether head (H) or tail (T) shows up, with
sample space S = {H,T}.

Yet another would consist of observing the gender of

the next child born at the local hospital, with S=
{F,M}.
 Sugata Munshi
Electrical Engg Deptt, JU
Many expts to which probability applies have more
than two o/cs.

The Multiplication Rule:

If an expt has M possible o/cs & another has N possible
o/cs, the 2 expts together have MN possible o/cs.

Proof: Let the possible o/cs of 1st expt be a1, a2,.aM &
those of 2nd expt be b1,b2,bN.

Then the possible o/cs of the 2 expts together, are

(a1,b1) (a1,b2) (a1,b3). (a1,bN)

(a2,b1) (a2,b2) (a2,b3). (a2,bN)
(a3,b1) (a3,b2) (a3,b3). (a3,bN)
..
..
(aM,b1) (aM,b2) (aM,b3). (aM,bN)

So, on the whole there are MN possible o/cs.

Generalization is as follows.
If there are K expts, each with possible o/cs
M1, M2,M3,.. MK respectively, the total number of
possible o/cs is
K
K M j
j 1

1. If we examine 3 fuses in sequence & note the result of

each examination, then an o/c for the entire expt is any
sequence of Ns & Ds of length 3, so
 Sugata Munshi
Electrical Engg Deptt, JU
S = {NNN, DNN,NDN, NND,DDN, NDD,DND, DDD}.

2. If we had tossed a coin 3 times, the sample space would be

S = {HHH, THH,THT, TTH,HHT, HTT,HTH, TTT}.

3. For the expt in which the genders of 3 successive newborn

children are observed, S would be obtd by replacing H with M &
T with F.

4. 2 fuel service stations are located at a certain

intersection. Each has 3 petrol pumps.

5. Consider expt # of pumps in use at a particular time

of day is determined for each of the stations.

6. An experimental o/c specifies how many pumps are in

use at the 1st station & how many are in use at the 2nd
one.

Coin-flipping:
S = { H, T} ; H head & T tail.
 Sugata Munshi
Electrical Engg Deptt, JU
Event:
When an expt is performed, a particular event A is
said to occur if the resulting experimental o/c is
contained in A.

Event or Event Set is any collection or subset of

possible o/cs in S.
An event is simple or elementary if it consists of exactly one
possible o/c & compound or composite if it consists of more
than one possible o/cs.

In general, exactly one simple event will occur, but

many compound events will occur simultaneously.

1. Example :
Fuel service stn example.

Event A----- On the whole six number of petrol pumps are

operating. (3,3)elementary event.
Event B----- On the whole four number of petrol pumps are
operating. (1,3), (2,2), (3,1)compound event.

Thus an elementary event cant be decomposed into simpler

events.

More than 1 choice of S for an expt. is possible.

Example :
Fuel service stn example.
 Sugata Munshi
Electrical Engg Deptt, JU

Suppose we are interested only in the situation when odd

no. of pumps of stn 1 and even no. of pumps of stn 2 are
working. Then,
S ={odd, even} = {1,2} ; {3,2}

Note:
S itself & the null (empty) set ( a set with no elements) are also
events of S.
Set s certain event &

null set impossible event.

Thus event subset of Sample Space.
An event is a set, all definitions & operations applicable to
sets will apply to events.

If a random expt. has S of C elements, the no. of events is 2 C

.

Tossing of a single coin:-

Events {H}, {T} , {H, T} & .

For example, if 2 events have no common o/c, they are mutually
exclusive.

Exhaustive Events: Several events are said to form an exhaustive

set, if at least one of the events must necessarily occur.
The complete group of all possible elementary events in a S, gives
an exhaustive set of events.
 Sugata Munshi
Electrical Engg Deptt, JU
Ex---
Rolling of a die:
A= { x1, x2 , x3, x4 , x5, x6 }
B= { odd, even }
1. Equally likely events:
O/cs of a random expt are said to be equally likely, if after
taking into consideration all relevant evidence, none of them can
be expected in preference to another.
Ex---- If a coin is unbiased or good or fair, i,e, it is in the form of
a perfect disc and is made of homogeneous material, the
occurrence of a head or tail are equally likely.
Ex--- If a die is unbiased, it has the exact shape of cube and is
made of homogeneous material. Consequently there is equal
likelihood of each face of the die showing up.
Ex--- Out of a full pack of 52 cards, two cards may be drawn in
C252 1326 ways. So there are 1326 equally likely o/cs.
2. Cases Favourable to an Event: Among all the possible o/cs of
a random expt, those cases , those cases which entail
occurrence of an event A are called cases favourable to A
Ex : In a fuel service station with 4 petrol pumps the no of
pumps in operation is described by
S= {0, 1,2 ,3 ,4}
If A= Even # of pumps operating= {2, 4}
 Sugata Munshi
Electrical Engg Deptt, JU
2 and 4 are o/cs favorable to A.
The following notations are used here:

1. The set of natural numbers, N={1,2,3,.}

2. The set of integers, Z={,3,2,1,0,1,2,3,}
3. The set of rational numbers Q .
4. The set of real numbers R.

Discrete & continuous sample spaces & Events:

1. A sample space is called discrete or countable if it contains
only finitely or infinitely many points that can be arranged into
a simple sequence E1, E2, E3 ..
Sample points outcomes

Tossing a six-faced die and observing the face showing up.

S is finite set of 6 elements.
S= { x1, x2 , x3, x4 , x5, x6 }
Such sample spaces are said to be discrete & finite.

In this expt. event even numbered face is up represented by

A= { x2 , x4 , x6 } is an example of discrete & finite event.
S can also be discrete & infinite.

In the experiment randomly select a +ve integer , S is the

countably infinite set {1,2,3,4, .}

Here an event select an odd integer represented by A= {

1,3,5,7,..} , is an example of discrete & infinite event. It has
countably infinite no. of elements.
 Sugata Munshi
Electrical Engg Deptt, JU
However , events on countably infinite S need not be compulsorily
countably infinite.

A={ 1,3,5,7 } possible event on above S.

Some expts. have continuous S , i.e. the sample points assume a

continuum of values.

Example: expt. Measure the resistance value x from an

assortment of 1 k 5% resistors S = { 0.95 k x 1.05
k } Continuous and finite S.
An event of interest might be to select a resistance value in the range
between 1.01 k to 1.04 k.

Example of cont. event on above S

A={ 1.01 k x 1.04 k} continuous and finite set.

Continuous S can also have discrete events. For example, A={

0.96 k, 0.98 k, 1.02 k} represents a discrete event. on a
continuous S.
1. A set of all possible positive numbers from 0 to constitute a
continuous & finite S.
 Sugata Munshi
Electrical Engg Deptt, JU
2. If the expt consists of measuring the lifetime of a car, then the
sample space consists of all nonnegative real numbers. That is,
S = [0, )

PROBABILITY
1. Classical or a priori Definition:
The classical definition of probability reduces the concept of
probability to the notion of equal probability (likelihood) of
events which becomes a basic assumption. Further, the
probability P(A) of an event A is determined a priori, w/o
experimentation.

Thus, if a random experiment has N number of exhaustive &

equally likely o/cs & NA of these are favourable to the event A,
NA
the Pr of event A is computed a priori as N .

NA
Thus, Pr( A) = N (1)

Thus 0 P( A ) 1
Example :
Throwing an unbiased die & observing the number on the face
that shows up.
S= { x1, x2 , x3, x4 , x5, x6 }
Probability of each number x showing up is P(x) =1/6
 Sugata Munshi
Electrical Engg Deptt, JU
That is, P(x1)=P(x2)=P(x3)=P(x4)=P(x5)=P(x6)=1/6
Example :
Throwing a pair of unbiased dice & observing sum of numbers
faced up.
S has 6 2 = 36 elements shown below.

Each possible o/c corresponds to having integral values from

2 to 12.

Consider event of getting a nine.

4

Table shows event can be achieved in 4 ways. Hence Pr(9)=

4/ 36 = 1/9.
Example:
 Sugata Munshi
Electrical Engg Deptt, JU
Two cards are drawn from a full pack of 52 cards. Find the
probability that (a) both are black cards & (b) one is heart and the
other is a diamond.
Solution:
2 cards may be drawn out of a pack of 52 cards in C252 1326 ways. So
there are 1326 exhaustive & equally likely o/cs.
(a) The pack contains only 26 black cards.
No. of cases favourable to the event both cards black is
C 226 325 . Hence reqd prob is p=325/1326=0.245
(b) each pack contains 13 spades, 13 clubs, 13 diamonds and 13
hearts.
So one diamond can be drawn in 13 ways & 1 heart can be drawn in
13.
Any of the heart cards can be combined with any of the diamond
cards to give a group of one heart & one diamond
Therefore the no. of cases favourable to the event is 13 x13= 169. So
reqd probability is
P = 169/1326= 0.127.
Defects of Classical definition:
1. It is based on the feasibility of having exhaustive and equally
likely o/cs. Not feasible in many practical settings.
Restricted to coin-tossing, die throwing & other games of chance.
2. Equally likely = equally probable.
 Sugata Munshi
Electrical Engg Deptt, JU
Definition tries to define probability in terms of equal probability.
How do we know whether the probabilities (likelihoods) are equal
unless they are measured? The definition is thus circular in
nature.
(iii) The formula suggests that it is important to be able to count elements
in sets. If sets are small, this is an easy task; however, if the sets are large,
this could be a difficult job.

2. Probability As A Measure of Frequency of Occurrence

(Statistical definition of probability) :
Statistical Regularity:
In order to be useful, a model must enable us to make
predictions about the future behavior of a system, and in
order to be predictable, a phenomenon must exhibit
regularity in its behavior. Many probability models in
engg are based on the fact that averages obtd in long
sequences of repetitions (trials) of random experiments
consistently yield approximately the same value.

This property is called statistical regularity.

That such regularity is reasonable is based purely on the fact that

many investigators of numerous physical expts have observed such
regularity.

Suppose: random expt. is performed n times.

 Sugata Munshi
Electrical Engg Deptt, JU
Certain event A occurs nA times.

Relative frequency () of occurrence of A is nA / n (average

number of successes)

tends to stabilize and approach a number Pr (A) as n

increases without limit.

probability of occurrence of A is

nA
P( A) = lim (2)
n n

In practice n is large < .

Quite clearly, since nA n, 0 P (A) 1.

 Sugata Munshi
Electrical Engg Deptt, JU
Example:

Probability of event {finding a defective PCB} 0.038

Notation:
1. The complement of an event A, denoted by Ac or A or , is the set of
all outcomes in that are not contained in A. Thus AC may be called an
event contrary to event A.

2. The union of two events A and B, denoted by and read A or B, is the

event consisting of all outcomes that are either in A or in B or in
both events. So the union includes outcomes for which both A and B
occur as well as outcomes for which exactly one occursthat is, all
outcomes in at least one of the events.

3. The intersection of two events A and B, denoted by and read A and

B, is the event consisting of all outcomes that are in both A and
B.
Since events have the properties of sets, when working with
events, intersection means "and", and union means "or". The probability
of intersection of A & B, P(AB), is sometimes shown by P(A,B)
or P(AB).

Axiomatic approach to Probability:

 Sugata Munshi
Electrical Engg Deptt, JU
Axiom # 1 : Pr (A) 0
Axiom # 2 : Pr (S) = 1
Axiom # 3 : A1 , A2 , A3 , .. be mutually exclusive (disjoint) events
defined on S.
Then,

Pr A1 A2 A3 ...........
= Pr A1 +Pr A 2 +Pr A3 +......... (3)
Pr. of union of non-overlapping events, is of pr. of individual events.
The axiom can also be stated using the following alternative symbolic
representation.

N N
Pr An = Pr(A n )
n 1 n=1

if A m An for all m n=1,2,3,....N,

with N possibly infinite.
Note: Let denote the null event (the event consisting of no outcomes
whatsoever). When A B= , A and B are said to be mutually exclusive or
disjoint events.

Observations:

1. 1st axiom states that probability cannot be negative. The smallest value
for P(A) is zero & if P(A)=0, then the event A will never happen.
2. 2nd axiom states that probability of whole sample space is equal to one,
i.e., 100%. Reason for this is that the sample space S contains all
possible o/cs of our random expt. Thus, the o/c of each trial always
belongs to S, i.e., the event S always occurs & P(S)=1.
3. 3rd axiom is probably the most interesting one. Basic idea is that if some
events are disjoint (i.e., there is no overlap betn them), then the
 Sugata Munshi
Electrical Engg Deptt, JU
probability of their union must be the of their probabilities. Another
way to think about this is to imagine the probability of a set as the
area of that set in the Venn diagram. If several sets are disjoint
such as the ones shown in Fig, then the total area of their U is the of
individual areas. The following example illustrates the idea behind the
3rd axiom.

1. Example
In a presidential election, there are four candidates. Call them A, B, C, &
D. Based on our polling analysis, we estimate that A has a 20% chance of
winning the election, while B has a 40% chance of winning. What is the
probability that A or B win the election?
Solution:
1. Notice that the events that {A wins}, {B wins}, {C wins} & {D wins} are
disjoint since more than one of them cant occur at the same time. For
example, if A wins, then B cant win.
2. From the 3rd axiom of probability, the probability of the union of two
disjoint events is the sum of individual probabilities.
3. Therefore,

P(A wins or B wins)= P({A wins})+P({B wins})=0.2+0.4=0.6

Example:

Throwing of a pair of unbiased dice.

Let event A={sum=5}
B = { 8< sum 10 } &
C= { sum 11 }
A, B, & C are mutually exclusive. On inspecting S it is found that
Pr (A) = 4/36 = 1/9
Pr (B) = 7/36
Pr (C) = 3/36 = 1/12
 Sugata Munshi
Electrical Engg Deptt, JU
Pr A B C Pr( A) Pr( B) Pr(C )
4 7 3 14

36 36 36 36

This result agrees with the probability value calculated

from inspection of S.

Example:
Suppose that the lifetime of a computer memory chip is measured, and we
find that the proportion of chips whose lifetime exceeds t decreases
exponentially at a rate Find an appropriate probability law.
Solution:
Let the sample space in this experiment be S= (0, ).

If we interpret the above finding as the probability that a chips lifetime

exceeds t decreases exponentially at a rate , we then obtain the following
assignment of probabilities to events of the form (t, ):

P t , e t ;t 0
where >0.
Note that the exponential is a number betn 0 & 1 for t >0, so Axiom I is
satisfied.
Axiom II is satisfied since P0, PS 1.

The probability that the lifetime is in the interval (r, s] is found by noting in
Fig. that ( r , s ] ( s, ) ( r , ) . So by Axiom III,

P r , P [( r ,s ]] P( s, )

By rearranging the above equation, we obtain,

P ( r ,s ] P [( r , )] P( s, ) e r e s
 Sugata Munshi
Electrical Engg Deptt, JU
We may thus obtain the probability that the lifetime is in any arbitrary
interval (r,s] in S.
 Sugata Munshi
Electrical Engg Deptt, JU

Pr( AB) Pr( A) Pr( B) Pr( A B)

 Sugata Munshi
Electrical Engg Deptt, JU

OTHER RULES OF PROBABILITY:

 Sugata Munshi
Electrical Engg Deptt, JU

1. Probability of the complement, i.e. of A not

occurring is denoted by

P ( A) 1 P ( A). (12)
2. Iff events A & B are s-independent, then,
P( A B) P( A B ) P( A) (13)
&
P( B A) P( B A) P( B) (13a)

That is, P(A) is unrelated to whether or not B

occurs and vice versa.

Prob. of any 1 of 2 s-independent events A & B occurring is

P( A B) P( A) P( B) P( A) P( B) (14)
For derivation of above equation consider,
 Sugata Munshi
Electrical Engg Deptt, JU

Here, either A or B, or A & B must work for

the system to work.
System success prob. = PS , then failure prob.= Pf = 1 PS .
1. Pf is jt. prob. of A & B failing, i.e.

Pf [1 P ( A)] [1 P ( B )]
1 P ( A) P ( B) P ( A) P ( B)
PS 1 Pf
P ( A B ) P ( A) P ( B ) P ( A) P ( B )

3. If A & B are mutually exclusive , i.e. A & B can not occur

simultaneously, then,
P(AB) = 0 (15)
and
P(A+B) = P(A) + P(B) (15a)
 Sugata Munshi
Electrical Engg Deptt, JU
4. If multiple, mutually exclusive probs. of outcomes
Bj jointly give a
probability of occurrence of A, then,

P( A) P AB j P A B j P( B j )
j j
(16)

6. From equation (7)

P( AB) P( A) P( B A) P( B) P( A B)
we have,

P( A) P B A
P A B (17)
P( B)
This is a simple form of Bayes theorem.
A more general form is,

P( A) P B A
P A B (18)

P B E j P( E j )
j

where,
E j is the jth event.