Lecture01 Intro Probability Theory

CSE-801: Stochastic Systems
Introduction to Probability Theory
Syed Ali Khayam School of Electrical Engineering & Computer Science National University of Sciences & Technology (NUST) Pakistan
Copyright Syed Ali Khayam 2009
Course Information
Lecture Timings:
Wednesdays: 5:30pm-7:20pm Fridays: 5:30pm-6:20pm
Office Hours
Wednesdays: 4:00pm-5:30pm Fridays: 4:00pm-5:30pm Office is located on top floor, last room on your left in the north wing
The course will be managed through Moodle:

lms.nseecs.edu.pk Course password: given in class
Textbook
Course Outline
Syllabus

Introduction to Probability Theory Functions of Random Variables Limits and Inequalities Stochastic Processes Prediction and Estimation Markov Chains and Processes (time permitting) Assorted Topics (time permitting)
Grading

Final Exam: 40% Midterm Exam: 30% Quizzes: 20% Homework Assignments: 10%
Course Outline
Grading

Final Exam: 40% Midterm Exam: 30% Quizzes: 20% Homework Assignments: 10%
Lots of extra credit for extra effort

We will also have a voluntary user study as well Anyone volunteering and seeing it through will get 10 extra credit points
Policies
Quizzes are announced and will take place at the start of Friday classes Exams will be closed book, but you will be allowed to bring an A4-sized cheat sheet to the exam
Late homeworks submissions will not be accepted

Strongest possible disciplinary action will be taken in case of plagiarism or cheating in exams, homeworks or quizzes It is mandatory to maintain at least 75% class attendance to sit in the Final Test
Credits and Acknowledgements
I would like to thank Dr. Garcia for providing online lecture notes on the textbooks website Throughout this course, I will be borrowing examples and explanations from the Stochastic Systems Course taught by Professor Hayder Radha at Michigan State University
Why this course?
Stochastic theory is an extension of probability theory This course on Stochastic theory will teach mathematical tools that are commonly-used in a variety of engineering, computer science and IT disciplines We will focus solely on performance modeling of phenomena observed in communications engineering Applications and examples will be provided as required
What will we cover in this lecture?
This lecture is intended to be an introduction to elementary probability theory We will cover:
Random Experiments and Random Variables Axioms of Probability Mutual Exclusivity Conditional Probability Independence Law of Total Probability Bayes Theorem
Definition of Probability
Probability: [m-w.org]
1 : the quality or state of being probable 2 : something (as an event or circumstance) that is probable 3 a (1) : the ratio of the number of outcomes in an exhaustive set of equally likely outcomes that produce a given event to the total number of possible outcomes (2) : the chance that a given event will occur b : a branch of mathematics concerned with the study of probabilities 4 : a logical relation between statements such that evidence confirming one confirms the other to some degree
10
Do you know which famous person was so opposed to probability theory that he said: God does not play dice with the universe.?
11
And do you know which famous person said: God does play dice with the universe. All the evidence points to him being an inveterate gambler, who throws the dice on every possible occasion.?
12
Definition of a Random Experiment
A random experiment comprises of:
A procedure An outcome
Procedure (e.g., flipping a coin)
Outcome (e.g., the value observed [head, tail] after flipping the coin)
Sample Space (Set of All Possible Outcomes)
13
Definition of a Random Experiment: Outcomes, Events and the Sample Space
An outcome cannot be further decomposed into other outcomes

{s1 = the value 1}, , {s6 = the value 6}
An event is a set of outcomes that are of interest to us

A = {s: such that s is an even number}
The set of all possible outcomes, S, is called the sample space

S = {s1, s2, s3, s4, s5, s6}
outcome
event
sample space
14
s1
s5 s6
s2
s4 s3
15

Example of a Random Experiment:
Experiment: Roll a fair die once and record the number of dots on the top face

S = {1, 2, 3, 4, 5, 6} A = the outcome is even = {2, 4, 6} B = the outcome is greater than 4 = {5, 6}
16
Axioms of Probability
Probability of any event A is non-negative: Pr{A} 0 The probability that an outcome belongs to the sample space is 1: Pr{S} = 1 The probability of the union of mutually exclusive events is equal to the sum of their probabilities: If A1 A2=, => Pr{A1 U A2} = Pr{A1} + Pr{A2}
17
Mutual Exclusivity
For mutually exclusive events A1, A2,, AN, we have:

N N
Pr
i 1
Ai
i 1
Pr Ai
A1
s1
s5
s6
A2
s2
s4 s3
Find Pr{A1 U A2} and Pr{A1}+Pr{A2} in the fair die example
S
Copyright Syed Ali Khayam 2008 18
Mutual Exclusivity
In general, we have:
Pr{A1 U A2} = Pr{A1} + Pr{A2} Pr{A1 A2}
s1
s5
s6
s2
s4
s3
S
Mutual Exclusivity: Example

Experiment: Roll a fair dice twice and record the number of dots on the top face:
S = { (1,1), (1,2), (1,3), (1,4), (1,5), (1,6), (2,1), (2,2), (2,3), (2,4), (2,5), (2,6), (3,1), (3,2), (3,3), (3,4), (3,5), (3,6), (4,1), (4,2), (4,3), (4,4), (4,5), (4,6), (5,1), (5,2), (5,3), (5,4), (5,5), (5,6), (6,1), (6,2), (6,3), (6,4), (6,5), (6,6)
20

Find the probability of the following events:
A1 = first roll gives an odd number A2 = second roll gives an odd number C = the sum of the two rolls is odd
21

Define three events:
A1 = first roll gives an odd number A2 = second roll gives an odd number C = the sum of the two rolls is odd Find the probability of C using probability of A1 and A2
22

A2
S={
(1,1), (1,2), (1,3), (1,4), (1,5), (1,6), (2,1), (2,2), (2,3), (2,4), (2,5), (2,6), (3,1), (3,2), (3,3), (3,4), (3,5), (3,6), (4,1), (4,2), (4,3), (4,4), (4,5), (4,6), (5,1), (5,2), (5,3), (5,4), (5,5), (5,6), (6,1), (6,2), (6,3), (6,4), (6,5), (6,6) }
A1
23

Pr{A1} = first roll gives an odd number = 18/36 = 1/2 Pr{A2} = second roll gives an odd number = 18/36 = 1/2 C = the sum of the two rolls is odd
24

C = the sum of the two rolls is odd Let C1 = first roll is odd and second is even= A1 C2 = first roll is even and second is odd= A1
C A1 A2 A1 A2
A2
A2
Since C1 and C2 are mutually exclusive:

Pr C Pr C 1 C2 Pr{C 1 } Pr{C 2 }
Pr A1 Pr A2 Pr A1 1 1 2
Pr A1 Pr A2 1 Pr A1 Pr A2
Pr A2
25
Conditional Probability
Given that event B has already occurred, what is the probability that event A will occur? Given that event B has already occurred, reduces the sample space of A
s1
s5 Event B has already occurred s6 => s2, s4, s3 cannot occur s2
s1
s5
s6
s2
s4
s3
s4
s3
26
Given that event B has already occurred, we define a new conditional sample space that only contains Bs outcomes The new event space for A is the intersection of A and B: EA|B = AB
s1
s5
Event B has already occurred
s1 s6
s5
s6
s2
s4 s3
s2
s4
s3
Whats missing here?

S|B = {s1, s5, s6} EA|B= A B = {s6}

27
Consider that the example below corresponds to an experiment where we throw a fair dice and record the number of dots on its face For this experiment, what is Pr {A|B} in the example below? Pr{A|B} = Pr{s6|B}= 1/3 We need to normalize all probabilities in a conditional sample space with Pr{B}
s1
s5 Event B has already occurred s2
s1
s6
s5
s6 s4
s2
s4 s3
S
s3
28
S|B = {s1, s5, s6}
We need to normalize all probabilities in a conditional sample space with Pr{B} Pr{s1|B} = Pr{s1}/Pr{B} = (1/6)/(1/2) = 1/3 Pr{s5|B} = Pr{s5}/Pr{B} = (1/6)/(1/2) = 1/3 Pr{s6|B} = Pr{s6}/Pr{B} = (1/6)/(1/2) = 1/3
s1
s1 s6
s5
s6 s4
s2
s4 s3
S
s3
S|B = {s1, s5, s6}

29
The probability of an event A in the conditional sample space is:

Pr A B Pr A Pr B B
For the dice example: Pr{A|B} = Pr{AB}/Pr{B} = Pr{s6}/Pr{B} = (1/6)/(1/2) = 1/3
s1
s1 s6
s5
s6 s4
s2
s4 s3
S
s3
30
S|B = {s1, s5, s6}
Independence
Two events are independent if they do not provide any information about each other
Pr A B
Pr A
In other words, the fact that B has already happened does not affect the probability of As outcomes
31
Independence
Note that
Pr A B
Pr A only when Pr A
Pr A Pr B
The above condition implies that
Pr A B
Pr A
Pr B A
Pr B
32
Independence
In general, for n mutually independent events, A1, , An, we have:

n n
Pr
i 1
Ai
i 1
Pr Ai
Pr Ak Ai1, Ai 2 ,
, Aip
Pr Ak ,
Aij
Ak
33
Independence: Example
Are events A and C independent?
Assume that all outcomes are equally likely
s2 s4 s6
s5
s1
s3
34
Are events A and C independent? Yes: Pr{A C} = Pr{s5} = 1/6 Pr{A}Pr{C} = (3/6)x(2/6) = 1/6
s2 s4 s6
s5 s1 s3
S
Are events A and B independent?
s2 s4 s6
s5 s1
s3
36
Are events A and B independent? NO: Pr{A B} = Pr{s5} = 1/6 Pr{A}Pr{B} = (3/6)x(3/6) = 1/4
s2 s4 s6
s5 s1 s3
S
Are events A and B independent?
s1
s5
s6
s2
s4 s3
38
Are events A and B independent? NO: Pr{A B} = Pr{} = 0 Pr{A}Pr{B} = (2/6)x(3/6) = 1/6 Recall that A and B are mutually exclusive
s1
s5
s6
s2
s4 s3
S
What you need to remember from what we have studied so far

1.
Outcomes, events and sample space:
outcome
event
sample space
2.
For mutually exclusive events A1, A2,, AN, we have:
Pr A1
3.
A2
Pr A1
Pr A2
In general, we have:
Pr A1
A2
Pr A1
Pr A2
Pr A1
A2
40
What you need to remember from what we have studied so far

4.
Conditional probability reduces the sample space:

Pr A B Pr A Pr B B
5.
Two events A and B are independent only if
Pr A B
Pr A
6.
For independent events:
Pr A
Pr A Pr B
41
Four Rules of Thumb from what we have studied so far

1. Whenever you see two events which have an OR relationship (i.e., event A or event B), their joint event will be their union, {A U B} Example: On a binary channel, find the probability of error? An error occurs when A: a 0 is transmitted and a 1 is received OR B: a 1 is transmitted and a 0 is received Thus probability of error is: Pr{A U B}
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
(See Appendix of this lecture for a explanation of the binary channel)
42

2. Whenever you see two events which have an AND relationship (i.e., both event A and event B), their joint event will be their intersection, {A B} Example: On a binary channel, find the probability that a 0 is transmitted and a 1 is received? An error occurs when A: a 0 is transmitted AND B: a 1 is received Thus probability of above event is: Pr{A B}
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
43

3. Whenever you see two events which have an OR relationship (i.e., A U B), check if they are mutually exclusive. If so, set Pr{A U B} = Pr{A} + Pr{B} Example: On a binary channel, find the probability of error? An error occurs when A: a 0 is transmitted and a 1 is received OR B: a 1 is transmitted and a 0 is received Thus probability of error is: Pr{error} = Pr{A U B} Are A and B are mutually exclusive?
Pr{R0|T0}
T0 R0
T1
Pr{R1|T1}
R1
44

3. Whenever you see two events which have an OR relationship (i.e., A U B), check if they are mutually exclusive. If so, set Pr{A U B} = Pr{A} + Pr{B} Example: On a binary channel, find the probability of error? An error occurs when A: a 0 is transmitted and a 1 is received OR B: a 1 is transmitted and a 0 is received Thus probability of error is: Pr{error} = Pr{A U B} YES! A and B are mutually exclusive; transmission of a 0 precludes the possibility of transmission of a 1, and vice versa. Therefore, we can set Pr{error} = Pr{A U B} = Pr{A} + Pr{B}
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
45

4. Whenever you see two events which have an AND relationship (i.e., A B), check if they are independent. If so, set Pr{A B} = Pr{A}Pr{B} Example: On a binary channel, find the probability that a 0 is transmitted and a 1 is received? A: a 0 is transmitted AND B: a 1 is received Probability of above event is: Pr{A B} Are A and B independent?
Pr{R0|T0}
T0
R0
T1
Pr{R1|T1}
R1
46

4. Whenever you see two events which have an AND relationship (i.e., A B), check if they are independent. If so, set Pr{A B} = Pr{A}Pr{B} Example: On a binary channel, find the probability that a 0 is transmitted and a 1 is received? A: a 1 is received AND B: a 0 is transmitted Probability of above event is: Pr{A B} Are A and B independent? NO! (See Appendix of this lecture for a more detailed explanation) Pr{A|B}=Pr{R1|T0} Pr{A}=Pr{R1}
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
47

4. Whenever you see two events which have an AND relationship (i.e., A B), check if they are independent. If so, set Pr{A B} = Pr{A}Pr{B} Example 2: On a binary channel, find the probability that a 0 is transmitted and a 1 is received? A: at time n+1, a 1 is received when a 0 is transmitted AND B: at time n, a 0 is received when a 1 is transmitted Probability of above event is: Pr{A B} Are A and B independent?
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
48
4. Whenever you see two events which have an AND relationship (i.e., A B), check if they are independent. If so, set Pr{A B} = Pr{A}Pr{B} Example 2: On a binary channel, find the probability that a 0 is transmitted and a 1 is received? A: at time n+1, a 1 is received when a 0 is transmitted AND B: at time n, a 0 is received when a 1 is transmitted Probability of above event is: Pr{A B} Are A and B independent? YES! Pr{A|B}=Pr{R1|T0}=Pr{A} => Pr{A B} = Pr{A}Pr{B} = Pr{R1|T0} Pr{R0|T1}
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
49
Partition of a Sample Space
B1, B2,, BN form a partition of a sample space we have:

S = B 1 U B2 U U BN Bi Bj = , i j
B2 B3
s1 B4 s2
s5 s6
B1
s4
s3
50
Total Probability
If B1, B2,, BN form a partition then for any event A

(A Bi) (A Bj) = , i j => A = (A B1) U (A B2) U U (A BN)
B2
s1
B1
s5
A
B3
A
B4 s2 s4
s6
s3
Total Probability
Thus event A can be expressed as the union of mutually exclusive events: A = (A B1) U (A B2) U U (A BN) => Pr{A} = Pr{A B1} + Pr{A B2} + + Pr{A BN}
B2
s1
B1
s5
A
B3
A
B4 s2 s4
s6
s3
Total Probability
If B1, B2,, BN form a partition then for any event A: Pr{A} = Pr{A B1} + Pr{A B2} + + Pr{A BN}
B2
s1
B1
s5
A
B3
A
B4 s2 s4
s6
s3
Total Probability
Using the definition of conditional probability: Pr{A| Bi} = Pr{A Bi} / Pr{Bi} => Pr{A Bi} = Pr{A| Bi} Pr{Bi}
B2
s1
B1
s5
A
B3
A
B4 s2 s4
s6
s3
The Law of Total Probability

The Law of Total Probability states:
If B1, B2,, BN form a partition then for any event A Pr{A} = Pr{A|B1} Pr{B1} + Pr{A|B2} Pr{B2} + + Pr{A|BN} Pr{BN}
B2 s1 B1 s5
A
B3
A
B4 s2 s4
s6
s3
55
Bayes Theorem
Based on the Law of Total Probability, Thomas Bayes decided to look at the probability of a partition given a particular event, the so-called inverse probability
B2
s1
B1
s5
A
B3
A
B4 s2 s4
s6
s3
Bayes Theorem
Based on the Law of Total Probability, Thomas Bayes decided to look at the probability of a partition given a particular event, the so-called inverse probability Pr{Bi|A} = Pr{A Bi} / Pr{A} Since Pr{A Bi} = Pr{A|Bi} Pr{Bi}, we obtain Pr{Bi|A} = Pr{A|Bi} Pr{Bi} / Pr{A}
B2 s1 B1 s5
A
B3
A
B4 s2 s4
s6
s3
57
Bayes Theorem
Pr{Bi|A} = Pr{A|Bi} Pr{Bi} / Pr{A} From the Law of Total Probability, we have:
Pr{A} = Pr{A|B1} Pr{B1} + Pr{A|B2} Pr{B2} + + Pr{A|BN} Pr{BN}
Pr Bi A Pr A Bi Pr Bi
N
Pr A B j Pr B j
j 1
Bayes Rule
B2 s1 B1 s5
A
B3
A
B4
s2
s4
s6
s3
Bayes Theorem
B1, B2,, BN are known as a priori events
these events are known before the experiment
Pr{Bi} is known as a priori probability Pr{Bi|A} is known as a posteriori probability
Experiment is performed; Event A is observed; now what is the probability that Bi has occurred
B2 s1 B1 s5
A
B3
A
B4
s2
s4
s6
s3
Bayes Theorem: Example
Bayes Theorem is best understood through a classical example of a memory-less binary channel shown below What we already know about this channel is:

A priori probabilities: Pr{T0}, Pr{T1} Channel probabilities: Pr{R0|T0}, Pr{R1|T0} = 1 - Pr{R0|T0}, Pr{R1|T1}, Pr{R0|T1} = 1 - Pr{R1|T1}
T0
Pr{R0|T0}
R0
T1
Pr{R1|T1}
R1
60
Given that we know Pr{T0} and Pr{T1}, we want to find:
Pr{Ti|Ri}: The probability that Ti was transmitted given that Ri has been received, i = 0, 1; or the probability of successful symbol transmission Pr{Ti|Rj}: The probability that Ti was transmitted given that Rj has been received, i j: or the probability of symbol error
T0
Pr{R0|T0}
R0
T1
Pr{R1|T1}
R1
61

Pr{correct transmission}: The probability that Ti was transmitted given that Ri has been received, i = 0, 1; or the probability of successful symbol transmission Pr{correct transmission} = Pr{ (T0 R0) U (T1 R1) } = Pr{T0|R0} Pr{R0} + Pr{T1|R1} Pr{R1}
Lets focus on this first
T0
Pr{R0|T0}
R0
T1
Pr{R1|T1}
R1
62

Lets first focus on finding Pr{T0|R0} From Bayes Rule, we know that Pr{T0|R0} = Pr{R0|T0} Pr{T0} / Pr{R0}
T0
Pr{R0|T0}
R0
T1
Pr{R1|T1}
R1
63
From Bayes Theorem, we know that Pr{T0|R0} = Pr{R0|T0} Pr{T0} / Pr{R0}
But we dont know this
We know these
Lets now focus on finding Pr{R0} in terms of what we already know
T0
Pr{R0|T0}
R0
T1
Pr{R1|T1}
R1
64
Lets now focus on finding Pr{R0} in terms of what we already know From the Law of Total Probability, we have Pr{R0} = Pr{R0|T0} Pr{T0} + Pr{R0|T1} Pr{T1}
We know all of these terms
T0
Pr{R0|T0}
R0
T1
Pr{R1|T1}
R1
65
Lets now focus on finding Pr{R0} in terms of what we already know From the Law of Total Probability, we have Pr{R0} = Pr{R0|T0} Pr{T0} + Pr{R0|T1} Pr{T1}
We know all of these terms
T0
Pr{R0|T0}
R0
T1
Pr{R1|T1}
R1
66

Pr{T0|R0} = Pr{R0|T0} Pr{T0} / Pr{R0} And Pr{R0} = Pr{R0|T0} Pr{T0} + Pr{R0|T1} Pr{T1} => Pr{T0|R0} = Pr{R0|T0} Pr{T0} / (Pr{R0|T0} Pr{T0} + Pr{R0|T1} Pr{T1})
All the terms in this expression are known We can now compute Pr{T0|R0}
T0
Pr{R0|T0}
R0
T1
Pr{R1|T1}
R1
67

Pr{T0|R0} = Pr{R0|T0} Pr{T0} / (Pr{R0|T0} Pr{T0} + Pr{R0|T1} Pr{T1})
Using similar computations, we can show that Pr{T1|R1} = Pr{R1|T1} Pr{T1} / (Pr{R1|T1} Pr{T1} + Pr{R1|T0} Pr{T0})
T0
Pr{R0|T0}
R0
T1
Pr{R1|T1}
R1
68

Pr{T0|R0} = Pr{R0|T0} Pr{T0} / (Pr{R0|T0} Pr{T0} + Pr{R0|T1} Pr{T1}) Pr{T1|R1} = Pr{R1|T1} Pr{T1} / (Pr{R1|T1} Pr{T1} + Pr{R1|T0} Pr{T0}) Plug the above values into the original equation to get Pr{correct transmission} = Pr{T0|R0} Pr{R0} + Pr{T1|R1} Pr{R1}
Pr{R0|T0}
T0
R0
T1
Pr{R1|T1}
R1
69
A Priori Probabilities
Pr{T0} = 0.45 Pr{T1} = 1 - Pr{T0} = 0.55 Pr{R0|T0} = 0.94 Pr{R1|T0} = 1 - Pr{R0|T0} = 0.06 Pr{R1|T1} = 0.91 Pr{R0|T1} = 1 - Pr{R1|T1} = 0.09
Channel probabilities:

T0
Pr{R0|T0}=0.94
R0
T1
Pr{R1|T1}=0.91
R1
70

Then: Pr{T0|R0} = Pr{R0|T0} Pr{T0} / (Pr{R0|T0} Pr{T0} + Pr{R0|T1} Pr{T1}) =0.94x0.45 / (0.94x0.45 + 0.09x0.55) = 0.8952 Pr{T1|R1} = Pr{R1|T1} Pr{T1} / (Pr{R1|T1} Pr{T1} + Pr{R1|T0} Pr{T0}) =0.91x0.55 / (0.91x0.55 + 0.06x0.45) = 0.9488
T0
Pr{R0|T0}=0.94
R0
T1
Pr{R1|T1}=0.91
R1
71
Lecture 1: Appendix A
Additional Examples and Explanations
72
Background on a Memoryless Binary Communication Channel

Memoryless: Bit transmission at time n+i, i>0 has no dependence on bit transmission at time n Binary: Only two symbols are transmitted, represented by T0 and T1 Prior Probabilities: Probabilities of T0 and T1 are calculated ahead of time from the data; Pr{T1}= 1 - Pr{T0} Crossover Probabilities: Pr{R0|T1} and Pr{R1|T0} are called crossover or biterror probabilities. These probabilities are also calculated ahead of time by sending training signals on the channel; Pr{R0|T0}=1-Pr{R1|T0}, Pr{R0|T1}=1-Pr{R1|T1}
Pr{R0|T0} T0 R0
T1 Pr{R1|T1}
R1
73
Example 1
On a binary channel, find the probability that a 0 is transmitted and a 1 is received? A: a 1 is received AND B: a 0 is transmitted Probability of above event is: Pr{A B} Are A and B independent?
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
74
Example 1
What is the sample space of our experiment?
Pr{R0|T0} T0 R0
T1
Spring 2008 Copyright Syed Ali Khayam 2008
Pr{R1|T1}
R1
75
Example 1
What is the sample space of our experiment? Sample Space, S = {(T0 R0), (T0 R1), (T1 R0), (T1 R1)}
Pr{R0|T0} T0 R0
T1
Spring 2008 Copyright Syed Ali Khayam 2008
Pr{R1|T1}
R1
76
Example 1
A: a 1 is received AND B: a 0 is transmitted Are A and B independent? A and B are independent when only when Pr{A B} = Pr{A}Pr{B}. So the main question is thefollowing: Is Pr{A B} = Pr{A}Pr{B}?
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
77
Example 1
A: a 1 is received AND B: a 0 is transmitted Is Pr{A B} = Pr{A}Pr{B} ? The LHS of the above equation is: Pr{A B} = Pr{A|B}Pr{B} = Pr{R1|T0}Pr{T0} The RHS is: Pr{A}Pr{B} = Pr{R1}Pr{T0} So the above question can be rephrased as: Is Pr{R1|T0}Pr{T0} = Pr{R1}Pr{T0} ?
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
78
Example 1
A: a 1 is received AND B: a 0 is transmitted Is Pr{R1|T0}Pr{T0} = Pr{R1}Pr{T0} ? We can get rid of Pr{T0} from both sides, so we are left with the following question: Is Pr{R1|T0} = Pr{R1} ?
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
79
Example 1
A: a 1 is received AND B: a 0 is transmitted Is Pr{R1|T0} = Pr{R1} ? It can be intuitively deduced that the above equality relation does not hold in general because from the figure below we can see that Pr{R1} should be a function of both Pr{R1|T0} and Pr{R1|T1}.
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
80
Example 1
A: a 1 is received AND B: a 0 is transmitted Is Pr{R1|T0} = Pr{R1} ?
It can be intuitively deduced that the above equality relation does not hold in general. Mathematically, we can show this by computing the Pr{R1}: Pr{R1} = Pr{ (T0 is txd AND R1 is recd) OR (T1 is txd AND R1 is recd)} Pr{R1} = Pr{ (T0 R1) U (T1 R1)}
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
81
Example 1
A: a 1 is received AND B: a 0 is transmitted Is Pr{R1|T0} = Pr{R1} ?
Pr{R1} = Pr{ (T0 R1) U (T1 R1)} Clearly, (T0 R1) and (T1 R1) are mutually exclusive events. => Pr{R1} = Pr{T0 R1} + Pr{T1 R1} or Pr{R1} = Pr{R1 T0} + Pr{R1 T1} which gives Pr{R1} = Pr{R1|T0}Pr{T0} + Pr{R1|T1}Pr{T1} Now we rephrase the question posed on the top of this slide as: Is Pr{R1|T0} = Pr{R1|T0}Pr{T0} + Pr{R1|T1}Pr{T1} ?
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
82
Example 1
A: a 1 is received AND B: a 0 is transmitted Is Pr{R1|T0} = Pr{R1|T0}Pr{T0} + Pr{R1|T1}Pr{T1} ? For the above relation to be satisfied, the following relationship must be satisfied: Pr{T0} = 1 => Pr{T1} = 1- Pr{T0} = 0
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
83
Example 1
A: a 1 is received AND B: a 0 is transmitted Final Result: Pr{T0} = 1 => A and B are independent So events A and B are independent when T0 is the only symbol being transmitted
Pr{R0|T0} T0 R0
T1
Pr{R1|T1}
R1
84

Lecture01 Intro Probability Theory

Caricato da

Informazioni sul documento

Descrizione originale:

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Lecture01 Intro Probability Theory

Caricato da

Copyright:

Formati disponibili

CSE-801: Stochastic Systems

Introduction to Probability Theory

Copyright Syed Ali Khayam 2009

Wednesdays: 5:30pm-7:20pm Fridays: 5:30pm-6:20pm

The course will be managed through Moodle:

lms.nseecs.edu.pk Course password: given in class

Copyright Syed Ali Khayam 2008

Copyright Syed Ali Khayam 2008

Copyright Syed Ali Khayam 2008

Lots of extra credit for extra effort

Copyright Syed Ali Khayam 2008

Late homeworks submissions will not be accepted

Copyright Syed Ali Khayam 2008

Credits and Acknowledgements

Copyright Syed Ali Khayam 2008

Why this course?

Copyright Syed Ali Khayam 2008

What will we cover in this lecture?

This lecture is intended to be an introduction to elementary probability theory We will cover:

Copyright Syed Ali Khayam 2008

Copyright Syed Ali Khayam 2008

Copyright Syed Ali Khayam 2008

Copyright Syed Ali Khayam 2008

Definition of a Random Experiment

A random experiment comprises of:

Procedure (e.g., flipping a coin)

Sample Space (Set of All Possible Outcomes)

Copyright Syed Ali Khayam 2008

Definition of a Random Experiment: Outcomes, Events and the Sample Space

An outcome cannot be further decomposed into other outcomes

An event is a set of outcomes that are of interest to us

The set of all possible outcomes, S, is called the sample space

Copyright Syed Ali Khayam 2008

Definition of a Random Experiment: Outcomes, Events and the Sample Space

Copyright Syed Ali Khayam 2008

Definition of a Random Experiment: Outcomes, Events and the Sample Space

Copyright Syed Ali Khayam 2008

Copyright Syed Ali Khayam 2008

For mutually exclusive events A1, A2,, AN, we have:

Find Pr{A1 U A2} and Pr{A1}+Pr{A2} in the fair die example

Mutual Exclusivity: Example

Copyright Syed Ali Khayam 2008

Mutual Exclusivity: Example

Copyright Syed Ali Khayam 2008

Mutual Exclusivity: Example

Copyright Syed Ali Khayam 2008

Mutual Exclusivity: Example

Copyright Syed Ali Khayam 2008

Mutual Exclusivity: Example

Copyright Syed Ali Khayam 2008

Mutual Exclusivity: Example

Since C1 and C2 are mutually exclusive:

Copyright Syed Ali Khayam 2008

s5 Event B has already occurred s6 => s2, s4, s3 cannot occur s2

Copyright Syed Ali Khayam 2008

Whats missing here?

S|B = {s1, s5, s6} EA|B= A B = {s6}

s5 Event B has already occurred s2

S|B = {s1, s5, s6}

s5 Event B has already occurred s2

S|B = {s1, s5, s6}

The probability of an event A in the conditional sample space is:

For the dice example: Pr{A|B} = Pr{AB}/Pr{B} = Pr{s6}/Pr{B} = (1/6)/(1/2) = 1/3