Notes on Non-cooperative Game Theory

Econ 8103, Spring 2009, Aldo Rustichini

Joseph Steinberg
May 12, 2009
Contents
1 Decisions Under Uncertainty 1
1.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Decision Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 Normal Form Games 6
2.1 Mixed vs. Correlated Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Dominance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3 Iterated Elimination and Rationalizability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.4 Nash Equilibria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5 Correlated Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.6 Zero-sum games (2 players) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.7 Perfect Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3 Extensive Form Games 21
3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Properties of extensive form games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3 Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4 Nash equilibria and renements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.4.1 Games of perfect information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.4.2 Games of imperfect information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.4.3 Sequential equilibria in extensive form games . . . . . . . . . . . . . . . . . . . . . . . 29
1 Decisions Under Uncertainty
1.1 Preliminaries
Before giving some basic denitions, we need to introduce some notation:
: Set of states.
T : Class of subsets of (-eld if is not nite, e.g., Borel sets)
X: Set of observable signals.
X: Class of subsets of X.
(X, X, P): Probability space; P is a probability measure on (X, X).
Denition 1.1. An experiment is a tuple c = (X, X, P

: ) where , P

is a probability
measure on (X, X).
The following is a simple example to illustrate these denitions.
1
Example 1.1. Let =
1
,
2
and let X = x
1
, x
2
. Let the experiment c be the tuple made up of X,
X = x
1
, x
2
, x
1
, x
2
and P

i , i = 1, 2 dened by
x
1
x
2

1
1

2
1
Figure 1: A simple experiment
Denition 1.2. A prior belief is a probability measure on (, T ).
Denition 1.3. A posterior belief ([x) () is a set of probabilities on ().
In general, given prior and an experiment c, the posterior ([x) is dened by Bayes rule:
([x) =
()P

(x)

()P

(x)
.
1.2 Decision Problems
Let A be an action set. A utility function is this context is a map u : A R. In other words, u(, a) is
the utility level if the state is and the action is a.
Denition 1.4. The set of mixed actions is (A, ), the set of probabilities on A .
Denition 1.5. The mixed extension of utility function u is the map u : (A, ) R dened by
u(, )
_
A
u(, a)(da).
If an agent observes a signal x X before acting, his choice of action will depend on the signal. This
naturally leads to another denition:
Denition 1.6. A policy is a map : X (A).
Let , c be given. For every policy , the expected utility is
_
X
P

(dx)
_

(d

[x)u(, (x)
where
P

(x) =
_

(x)(d

).
Example 1.2. Let =
1
,
2
and X = x
1
, x
2
as in example 1 above with , > 1/2. Let the prior
be (
1
) = (
2
) = 1/2. Let the utility function be the guessing utility:
u(
1
, a
1
) = u(
2
, a
2
) = 1
u(
1
, a
2
) = u(
2
, a
1
) = 0
If the signal is x
1
, ([x
1
) =
_

1+
,
1
1+
_
. Since > 1/2, the optimal policy is to guess
1
, i.e., always
choose a
1
. Then (x
1
) = (1, 0). Similarly, if the signal is x
2
, the optimal policy is to guess
2
((x
2
) = (0, 1)).
Therefore the expected utility under the optimal policy is
_
X
P

(dx)
_

(d[x)u(, (x) =
_
1
2
+
1
2
(1 )
_ _

1 +
(1) +
1
1 +
(0)
_
+
_
1
2
+
1
2
(1 )
_ _
1
1 +
(0) +

1 +
(1)
_
2
which simplies to
1
2
+
1
2
.
We can see that this policy is indeed optimal, because always guessing
1
or always guessing
2
both give
expected utility of
1
2
, while guessing
2
given x
1
and
1
given x
2
gives
1
2
(1 ) +
1
2
(1 ). This leads us to
a discussion of the value of information. In other words, which experiments are better?
Denition 1.7. Given u and , the value of P (probability given by an experiment c) is
V (P, u, ) = max
:X(A)
_
X
P

(dx)
_

(d[x)u(, (x).
Lemma 1.1. (Ill cross that bridge when I come to it.)
V (P, u, ) =
_
X
P

(dx) max
(A)
_

(d[x)u(, ).
Proof. See the solution for problem set 1.
Given and u as specied in the two examples above, V (P, u, ) is high when and are both close
to 1. If both and are less than 1/2, the optimal policy is reversed. In this case, we would want and
close to zero. However, the power of such comparison is limited since it depends on the specication of
and u. This brings us to the next denition.
Denition 1.8. Fix (, T ). Experiment c
2
is Bayesian more informative than experiment c
1
if
V (P
2
, u, ) V (P
1
, u, ), A, u, .
Example 1.3. Given the same and X as in the previous example, dene c
N
where P
N
is given by
x
1
x
2

1
1/3 2/3

2
1/3 2/3
Figure 2: An experiment that conveys no information at all
This experiment gives no information (hence the N); for each signal the states occur with equal
probability. The value of c
N
is
V
N
(P
N
, u, ) = max
:X(A)

1
3
u(
1
, (x
1
)) +

2
3
u(
2
, (x
1
)) +
2
1
3
u(
1
, (x
2
)) +
2
2
3
u(
2
, (x
2
)).
Also dene c
F
where P
F
is given by
x
1
x
2

1
1 0

2
0 1
Figure 3: A fully informative experiment
This experiment gives full information (hence the F); the signals are perfectly correlated with the
states. The value of c
F
is
V
F
(P
F
, u, ) = max
:X(A)

1
u(
1
, (x
1
) +
2
u(
2
, (x
2
)).
Clearly c
F
is more informative than c
N
, so we should have V
F
V
N
for all A, u, . Let

N
and

F
denote
the optimal policy functions associated with the two experiments. Then
V
N
(P
N
, u, ) =

1
3
u(
1
,

N
(x
1
)) +

2
3
u(
2
,

N
(x
1
)) +
2
1
3
u(
1
,

N
(x
2
)) +
2
2
3
u(
2
,

F
(x
2
))
3
and
V
F
(P
F
, u, ) =
1
u(
1
,

F
(x
1
) +
2
u(
2
,

F
(x
2
)).
So it must be that

F
(x
1
) argmax
(A)

1
u(
1
, ).
This implies that

1
u(
1
,

F
(x
1
))

1
3
u(
1
,

N
(x
1
)) +
2
1
3
u(
1
,

N
(x
2
)).
Similarly,

2
u(
2
,

F
(x
2
))

2
3
u(
2
,

N
(x
1
)) +
2
1
3
u(
2
,

N
(x
2
)).
Thus V
F
V
N
for all A, , u, so c
F
is Bayesian more informative than c
N
.
This denition gives us a basis for comparing the informational value of an experiment. But when is c
2
better than c
1
?
Denition 1.9. Experiment c
1
is a randomization of experiment c
2
if T : X X such that
P

(dx
1
) =
_
x
2
P

(dx
2
)T(x
2
, dx
1
).
That is, to get x
1
: First, given you observe x
2
. Second, given x
2
, you randomize over x
1
.
Example 1.4. Let c
2
= (R, B(R), N(,
2

2
) : R). Then X
2
is the random variable produced by c
2
.
Let c
1
= (R, B(R), N(,
2
) : R), so X
1
is the random variable produced by c
1
. Then X
1
= X
2
+ Y
where Y N(0, (1
2
)
2
) because var(X
1
) = var(X
2
) +var(Y ). Thus c
1
is a randomization of c
2
.
Theorem 1.1. If experiment c
1
is a randomization of c
2
then c
2
is Bayesian more informative than c
1
.
Proof. Since c
1
is a randomization of c
2
, T : X (X) such that
P

(dx
1
) =
_
X
P

(dx
2
)T(x
2
, dx
1
).
We want to show that c
2
is Bayesian more informative than c
1
, i.e., V
2
(P, , u) V
1
(P
1
, , u), , u.
Lets look at V
1
. We can rewrite it as follows.
V
1
(P
1
, , u) = max
(x1):X(A)
_
X
P

(dx
1
)
_

(d[x
1
)u(, (x
1
))
= max
(x1):X(A)
_
X
Pr(x
1
, )u(, (x
1
))ddx
1
= max
(x1):X(A)
_
X
__
X
Pr(x
1
, x
2
, )dx
2
_
u(, (x
1
))ddx
1
= max
(x1):X(A)
_
X
__
X
P

(x
2
)T(x
1
, x
2
)([x
2
)dx
2
_
u(, (x
1
))ddx
1
= max
(x1):X(A)
_
X
P

(x
2
)
__

([x
2
)
__
X
u(, (x
1
))T(x
1
, x
2
)dx
1
_
d
_
dx
2
Now lets look at V
2
. By lemma 1.1, we have
V
2
(P
2
, , u) = max
(x2):X
_
X
P

(x
2
)
_

(d[x
2
)u(, (x
2
))
=
_
X
P

(x
2
)
_
max
(A)
_

([x
2
)u(, )d
_
dx
2
4
So we really just need to show that
max
(A)
_

([x
2
)u(, )d
_

([x
2
)
__
X
u(, (x
1
))T(x
1
, x
2
)dx
1
_
d.
We can really think about the RHS as
#X

i=1
w
i
_

([x
2
)u(,
i
)d
where

#X
i=1
w
i
= 1. It is clear that
max
(A)
_

([x
2
)u(, )d
#X

i=1
w
i
_

([x
2
)u(,
i
)d
so V
2
V
1
.
Example 1.5. Let =
1
,
2
and let X = x
1
, x
2
. Dene the experiment c = (X, X, P) where P is
given by
x
1
x
2

1
1

2
1
Figure 4: The usual experiment
Similarly, dene c

= (X, X, P

) where P

is given by
x
1
x
2

1
1

2
1
Figure 5: The opposite experiment
Dene the map T by
x
1
x
2
x
1
0 1
x
2
1 0
Figure 6: The map T randomizes over x
1
Then P = P

+T, i.e.,
_
1
1
_
=
_
1
1
_ _
T
x
1
,x
1 T
x
1
,x
2
T
x
2
,x
1 T
x
2
,x
2
_
Then c is a randomization of c

. But we also have

_
1
1
_
=
_
1
1
_ _
T
x
1
,x
1 T
x
1
,x
2
T
x
2
,x
1 T
x
2
,x
2
_
so c

is also a randomization of c. By Theorem 1, the two experiments are equally informative: V (P, u, ) =
V (P

, u, ) for all A, u, . This brings us back to the discussion of how having and close to 1 and having
them close to 0 both yield high expected utility.
5
2 Normal Form Games
Denition 2.1. A normal form game is a tuple ((A
i
)
iI
, (u
i
)
iI
) where
I = 1, . . . , n is the set of players.
i I, A
i
is player is action set.
A =
iI
A
i
.
i I, u
i
: A R.
For the rest of the section, let A
i
be nite for all i I.
Denition 2.2. The mixed extension of a normal form game is a tuple ((S
i
)
iI
, (u
i
)
iI
) where
i I, S
i
(A
i
) = (a
i
1
, . . . , a
i
Ki
) = (s
i
(a
i
1
), . . . , s
i
(a
i
Ki
) : s
i
(a
i
k
) 0, k = 1, . . . , K
i
,

Ki
k=1
s
i
(a
i
k
) =
1.
S =
iI
S
i
.
i, s
i
S
i
, s
i
(a
i
j
) 0 and

Ki
j=1
s
i
(a
i
j
) = 1.
i I, u
i
: S R is dened by u
i
(s
1
, . . . , s
n
)

aA
u
i
(a) Pr
s
(a) where Pr
s
(a) =

n
j=1
s
j
(a
j
).
Before we get into the meat of the section, it will be helpful to establish the following result.
Theorem 2.1. u
i
: S R is a continuous function.
Proof. It suces to show that u is linear in each of its n arguments. Let s = (s
1
, . . . , s
n
) S. By the
denition above,
u
i
(s
1
, . . . , s
n
) =

aA
u
i
(a)
n

j=1
s
j
(a
j
).
For any k I, we can write this as
u
i
(s) =

aA
u
i
(a)s
k
(a
k
)
n

j=k
s
j
(a
j
)
=

a
k
A
k
s
k
(a
k
)

a
k
A
k
u
i
(a
k
, a
k
)

j=k
s
j
(a
j
)
So holding s
k
constant, u
i
is linear in s
k
. Since u
i
is linear in each of its arguments it is continuous.
2.1 Mixed vs. Correlated Strategies
Denition 2.3. If A is the set of action proles then the set of correlated strategies is (A). In other
words, (A), is a correlated strategy.
Let s = (s
1
, s
1
, . . . , s
n
) S =
iI
S
i
. We have dened Pr
s
(A) by Pr
s
(a) =

iI
s
i
(a
i
). We say
that Pr
s
is the probability induced on A by the mixed strategy s. Because Pr
s
(A), s S, Pr
s
is a
correlated strategy for all s S. Note that Pr
s
: s S (A). So while Pr
s
is a correlated strategy
s S, there are other correlated strategies that are not induced by mixed stragies. For any correlated
strategy (a), we can think of (a
i
) as a belief of player i on A
i
. It is important to recognize that if
(A
i
) player is belief on A
i
,
argmax
s
i
S
i

a
i
A
i
u
i
(s
i
, a
i
)(a
i
) = argmax
s
i
S
i

a
i
A
i
[Au
i
(s
i
, a
i
) +B](a
i
), A > 0, B R.
6
L R
U s
2
(L)s
3
(U) s
2
(R)s
3
(U)
D s
2
(L)s
3
(D) s
2
(R)s
3
(D)
Figure 7: Probability distribution induced by a mixed strategy prole
Example 2.1. Let I = 1, 2 and A = T, B L, R. Let Pr
s
1
,s
2 be a correlated strategy. Then
Pr
s
1
,s
2(T, L) = s
1
(T)s
2
(L). Suppose we add another player to the game with A
3
= U, D. Then from
player is point of view, a belief on A
i
is Pr
s
2
,s
3 . The table below contains player is beliefs about the
probability of each a
i
A
i
. Note that Pr
s
2
,s
3(L, U)/ Pr
s
2
,s
3 (L, D) = Pr
s
2
,s
3 (R, U)/ Pr
s
2
,s
3 (R, D).
Examining these ratios is a way of testing whether or not a correlated strategy could be a mixed
strategy. The next example illustrates this concept. Consider the correlated strategy in gure 8. Then
= ((T, L), (B, L), (T, R), (B, R) = (1/2, 0, 0, 1/2). It is clear that (T, L)/(B, L) ,= (T, R)/(B, R).
Thus (A) but , Pr
s
: s S, so is a correlated strategy but not a mixed strategy.
L R
T 1/2 0
B 0 1/2
Figure 8: A correlated strategy that is not induced by a mixed strategy
Theorem 2.2. The set of probabilities induced by mixed strategies is closed.
Proof. Take a sequence Pr
sn
that converges to some (A). We want to show Pr
s
: s S. S
is compact, so a convergent subsequence s
n
k
such that s
n
k
s S. Then Pr
sn
k
Pr
s
, so Pr
s
=
Pr
s
: s S.
However, Pr
s
: s S is not convex. To see this, we can obtain the correlated strategy in gure 8 by
taking a linear combination of the probabilities induced by the following mixed strategy proles in gure 9.
L R
T 1 0
B 0 0
L R
T 0 0
B 0 1
Figure 9: Two probability distributions induced by mixed strategies where any convex combination is not
induced by any mixed strategy
2.2 Dominance
I will start this section with an example.
Example 2.2. Let the payos for player 1 in a game be as described in the following gure. Will player 1 ever
L R
T 3 0
M 0 3
B 1 1
Figure 10: A game that illustrates dominance
choose to play B? There is no single pure strategy that is always better than B, but s
1
= (1/2, 1/2/0) S
1
gives expected utility of 3/2 regardless of which strategy player 2 chooses. So s
1
seems to be better than B.
This brings us to the next denition.
Denition 2.4. A mixed strategy s
i
S
i
strictly dominates a
i
A
i
if u
i
(s
i
, a
i
) > u
i
(a
i
, a
i
) for all
a
i
= (a
1
, . . . , a
i1
, a
i+1
, . . . , a
n
) A
i
. For now, we will refer to this version of strict dominance as D
1
,
i.e., s
i
D
1
a
i
.
7
We will now introduce two alternative denitions of strict dominance.
Denition 2.5. s
i
D
2
a
i
if s
i
S
i
, u
i
(s
i
, s
i
) > u
i
(a
i
, s
i
).
Denition 2.6. s
i
D
3
a
i
if (A
i
), u
i
(s
i
, ) > u
i
(a
i
, ), where u
i
(a
i
, )

a
i
A
i u
i
(a
i
, a
i
)(a
i
).
If it wasnt obvious already, these three denitions are equivalent. This brings us to the next theorem.
Theorem 2.3. s
i
D
1
a
i
s
i
D
2
a
i
s
i
D
3
a
i
.
Proof. S
i
=
j=i
S
j
. We have dened Pr
s
(a)

n
j=1
s
j
(a
j
), so Pr
s
i (a
i
)

j=i
s
j
(a
j
). We know that
S
i
(A
i
) and

Ki
j=1
s
i
(a
i
j
) = 1 for all i I. Then Pr
s
i (A
i
) for all s
i
S
i
. This implies that
s
i
D
3
a
i
s
i
D
2
a
i
.
For any a
i
A
i
, s
i
S
i
with probability 1 on a
i
and probability zero for all other b
i
A
i
.
Thus u
i
(, a
i
) = u
i
(, s
i
). This implies that s
i
D
2
a
i
s
i
D
1
a
i
.
Suppose s
i
D
1
a
i
. Then a
i
A
i
, u
i
(s
i
, a
i
) > u
i
(a
i
, a
i
). Let (A
i
). Then
u
i
(s
i
, a
i
)(a
i
) u
i
(a
i
, a
i
)(a
i
), a
i
A
i
with strict inequality when (a
i
) > 0. Adding up over all a
i
, this implies that u
i
(s
i
, ) > u
i
(a
i
, ). Thus
s
i
D
1
a
i
s
i
D
3
a
i
. Therefore s
i
D
1
a
i
s
i
D
2
a
i
s
i
D
3
a
i
.
Example 2.3. Dene a game with payos for player 1 that are similar to the previous example. Let p be
L R
T 3 0
M 0 3
B x x
Figure 11: A game with strictly dominated actions
the probability that player 2 plays L. If x < 3/2, B is strictly dominated by the mixed strategy (1/2, 1/2, 0)
for all possible p. However, at x = 3/2, at p = 1/2 there is no mixed strategy that strictly dominates B.
We have now dened what it means for a strategy to dominate an action. It seems reasonable to believe
that an action that is dominated will never be played. This bring us to the next denition.
Denition 2.7. An action a
i
A
i
is never a best response (NBR) if (A
i
) such that u
i
(a
i
, )
u
i
(b
i
, ), b
i
A
i
. Equivalently, a
i
is NBR if (A
i
), b
i
A
i
) such that u
i
(b
i
, ) > u
i
(a
i
, ).
Theorem 2.4. If a
i
A
i
is strictly dominated then it is never a best response (D NBR).
Proof. If a
i
is strictly dominated then s
i
S
i
such that a
i
A
i
, u
i
(s
i
, a
i
) > u
i
(a
i
, a
i
). Take any
(A
i
). Then
u
i
(s
i
, a
i
)(a
i
) u
i
(a
i
, a
i
)(a
i
)a
i
with strict inequality if (a
i
) > 0. Since (a
i
) > 0 for some a
i
,

a
i
A
i
u
i
(s
i
, a
i
)(a
i
) >

a
i
A
i
u
i
(a
i
, a
i
)(a
i
).
There must be b
i
support(s
i
) A
i
such that

a
i
A
i
u
i
(b
i
, a
i
)(a
i
)

a
i
A
i
u
i
(s
i
, a
i
)(a
i
).
Then

a
i
A
i
u
i
(b
i
, a
i
)(a
i
) >

a
i
A
i
u
i
(a
i
, a
i
)(a
i
)
so u
i
(b
i
, ) > u
i
(a
i
, ). Thus a
i
is never a best response.
8
Theorem 2.5. If a
i
A
i
is never a best response it is strictly dominated (NBR D).
Proof. We will make use of the theorem of the alternative, which says that for any matrices A, B, C and
vectors a, b, c,
x such that
_
_
Ax a
Bx b
Cx = c
_
_
0, 0, such that
_
_
A +B +C = 0
a +b +c 0
(a +c) +b +c > 0
_
_
Dene the #A
i
by #A
i
matrix U as
U =
_

_
u
i
(a
i
1
, a
i
1
) . . . u
i
(a
i
#A
i
, a
i
1
)
.
.
.
.
.
.
u
i
(a
i
1
, a
i
#A
i
) . . . u
i
(a
i
#A
i
, a
i
#A
i
)
_

_
Let b
i
A
i
. Dene the #A
i
-length column vector u as
u =
_

_
u
i
(b
i
, a
i
1
)
.
.
.
u
i
(b
i
, a
i
#A
i
)
_

_
So b
i
is never a best response if
=
_

1
.
.
.

#A
i
_

_ (A
i
)
such that
T
U
T
ue
T
, where e is a #A
i
-length column vector of ones. We can also express dominance in
matrix form. b
i
is strictly dominated if
s
i
=
_

_
s
i
(a
i
1
)
.
.
.
s
i
(a
i
#A
i
)
_

_
such that Us
i
u, Is
i
0 and e
T
s
i
= 1, where I is the #A
i
-dimension identity matrix and e is again a
#A
i
-length column vector of ones. Note that I have used the conditions Is
i
0 and e
T
s
i
= 1 to imply that
s
i
S
i
= (A
i
) is a #A
i
-length vector s
i
. This technique is what will allow us to apply the theorem of the
alternative.
Let b
i
A
i
such that b
i
is never a best response. Suppose for contradiction that b
i
is not strictly
dominated. Then a #A
i
-length column vector s
i
such that
_
_
Us
i
u
Is
i
0
e
T
s
i
= 1
_
_
.
By the theorem of the alternative, this implies that
0, 0, such that
_
_

T
U +I +e
T
= 0

T
u +(0) +(1) 0

T
(u + 1) +(0) +(1) > 0
_
_
.
Suppose = 0. Then we have I + e
T
= 0 and > 0. This contradicts 0, so it must be that ,= 0.
Since 0 and ,= 0, we can normalize so that (A
i
) (and normalize and as well. We can use
the above system of equations to get the following:

T
U +e
T
0 (*)

T
u + 0 (**)
By (*),
T
U e
T
. By (**),
T
u. This implies that
T
U
T
ue
T
. This contradicts the fact that
b
i
is never a best response. Therefore it must be that b
i
is strictly dominated.
The last two theorems establish an important (and intuitive) result: D NBR.
9
2.3 Iterated Elimination and Rationalizability
Denition 2.8. An iterated elimination of strictly dominated actions (IESDA) is a sequence A
1
, A
2
, . . . , A
T
such that
1. k = 1, . . . , T, A
k
=
iI
A
i
k
.
2. k = 1, . . . , T and i I, A
k+1
A
i
k
.
3. i I, A
i
1
= A
i
(initial action set).
4. k = 1, . . . , T and i I, a
i
A
i
k
A
i
k+1
only if s
i
(A
i
k
) such that a
i
A
i
k
, u
i
(s
i
, a
i
) >
u
i
(a
i
, a
i
) (elimination rule).
The elimination rule means that a
i
is eliminated at step k if it is dominated by a mixed strategy s
i
over
A
i
k
in the game with action proles in A
k
.
Example 2.4. Consider the following game:
L R
T 3, 0 0, 1
M 0, 0 3, 1
B 1, 1 1, 0
Figure 12: A game that illustrates IESDA
In this game, A
1
1
= T, M, B and A
2
1
= L, R. From example 6, we know that s
1
= (1/2, 1/2, 0)DB.
So B is eliminated from player 1s set of actions. Given that player 2 knows this, s
2
= (0, 1)DL. Thus L is
eliminated from player 2s action set. Finally, given that player 2 will only play R, M dominates T. Thus
player 1 will eliminate T as well. This leads to a nal action set A
T
= M R. Since each player only
has one action now, no more actions can be eliminated. This is referred to as a complete IESDA. Note
that we have need to allow dominance by mixed strategies for this to work; neither T nor M alone strictly
dominates B.
Denition 2.9. An action a
i
A
i
is rationalizable if there exists a vector (R
1
, . . . , R
n
) such that
1. a
i
R
i
.
2. j, R
j
A
j
.
3. j, b
j
R
j
, (b
j
) (A
j
) with support((b
j
)) R
j
such that b
j
is a best response to (b
j
), i.e.,
u
i
(b
j
, ) u
i
(a
j
, ), a
j
A
j
.
This denition implies that if the vector (R
1
, . . . , R
n
) rationalizes a
i
R
i
, then it also rationalized
a
k
R
k
, k. Note that u
i
(b
j
, ) u
i
(a
j
, ), a
j
A
j
, NOT just a
j
R
j
. Given (b
j
), b
j
must give at
least as much expected utility as ALL other actions in player js action set.
Example 2.5. Consider the following game:
L R
T 3, 0 0, 1
M 0, 0 3, 1
B 1, 1 1, 0
Figure 13: A game that illustrates rationalizability
From observation we can see that (R
1
, R
2
) = (M), (R).
Theorem 2.6. Suppose (R
1
, . . . , R
n
) and (T
1
, . . . , T
n
) are rationalizable sets. Then
(R
1
T
1
, . . . , R
n
T
n
)
is also rationalizable.
10
Proof. Consider any a
i
R
i
. Since (R
1
, . . . , R
n
) rationalizes a
i
, we know there exists (a
i
) (A
i
) with
support((a
i
)) R
i
such that a
i
is a best response to (a
i
). Then (R
1
T
1
, . . . , R
n
T
n
) also rationalizes
a
i
with this same (a
i
) (since the support is still just a subset of R
i
, the extra elements in T
i
dont
matter). The same logic works for any a
i
T
i
.
Theorem 2.7. There is only one maximal set of rationalizable actions (i.e., maximal under inclusion).
Proof. Suppose there are two dierent maximal sets. By the previous theorem, their union is also a ratio-
nalizable set. Contradiction.
The next few theorems establish the relationship between IESDA and rationalizability.
Theorem 2.8. Let R be a set of rationalizable actions. Let (A
1
, . . . , A
T
) be an IESDA. Then i I,
R
i
A
i
T
.
Proof. By induction on m, the stage of elimination.
Step 1: i I, R
i
A
i
1
= A
i
by denition.
Step 2: Assume i I, R
i
A
i
m
. We want to show that i I, R
i
A
i
m+1
. This will be true if and
only i I, a
i
R
i
is not eliminated at step m of the IESDA.
Since R is a set of rationalizable actions, a
i
R
i
implies that (a
i
) (A
i
) with support((a
i
))
R
i
such that
b
i
A
i
, u
i
(a
i
, ) u
i
(b
i
, ).
Because R
i
A
i
m
, i I, this implies that (a
i
) (A
i
) with support((a
i
)) A
i
m
such that
b
i
A
i
, u
i
(a
i
, ) u
i
(b
i
, ).
By equivalence of D and NBR, this means that s
i
(A
i
) such that
a
i
A
i
m
, u
i
(s
i
, a
i
) > u
i
(a
i
, a
i
).
Then a
i
is not eliminated at step m of the IESDA. Thus R
i
A
i
m+1
.
The induction implies that R
i
A
i
T
. This completes the proof.
Note that this theorem implies that for ANY set of rationalizable actions R, R A
T
. Then it must be
true for the maximal set of rationalizable actions.
Theorem 2.9. Let (A
1
, . . . , A
T
) be a complete IESDA. Let (R
1
, . . . , R
n
) be the maximal set of rationalizable
actions. Then A
T
R =
iI
R
i
.
Proof. It is enough to prove that A
T
=
iI
A
i
T
is a rationalizable set, since it would have to be a subset
of R. A
T
is the set of actions surviving complete IESDA, so i I and a
i
A
i
T
, s
i
(A
i
T
) such that
a
i
A
i
T
, u
i
(s
i
, a
i
) > u
i
(a
i
, a
i
).
We want to show that i and a
i
A
i
T
, (a
i
) (A
i
) with support((a
i
)) A
i
T
such that
u
i
(a
i
.) u
i
(b
i
, ), b
i
A
i
. If s
i
(A
i
) instead of s
i
(A
i
T
), the conclusion would follow from
equivalence of D and NBR. We know that A
i
T
A
i
T1
, and every element of A
i
T1
A
i
T
is strictly dominated
by mixed strategy of actions in A
i
T1
. So if s
i
(A
i
T
), it must be that s
i
(A
i
T1
either.
By induction, s
i
(A
i
) such that a
i
A
i
T
, u
i
(s
i
, a
i
) > u
i
(a
i
, a
i
). Thus equivalence of D and
NBR completes the proof.
Theorem 2.10. Let A
T
be the set of action proles surviving a complete IESDA. Then A
T
= R, where
R =
iI
R
i
is the maximal set of rationalizable actions. Further, A
T
is unique, so it does not depend on
the process of IESDA (as long as its complete).
Proof. By the previous two theorems, we have R A
T
and A
T
R. Thus A
T
= R. By theorem 2.7, R is
unique. Thus A
T
is also unique.
The last part of this section concerns weakly dominated actions and iterated elimination of weakly
dominated actions.
11
Denition 2.10.
s
i
S
i
weakly dominates a
i
A
i
if
1. a
i
A
i
, u
i
(s
i
, a
i
) u
i
(a
i
, a
i
).
2. b
i
A
i
such that u
i
(s
i
, b
i
) > u
i
(a
i
, b
i
).
Denition 2.11. An iterated elimination of weakly dominated actions (IEWDA) is a sequence
A
1
, A
2
, . . . , A
T
such that
1. k = 1, . . . , T, A
k
=
iI
A
i
k
.
2. k = 1, . . . , T and i I, A
k+1
A
i
k
.
3. i I, A
i
1
= A
i
(initial action set).
4. k = 1, . . . , T and i I, a
i
A
i
k
A
i
k+1
only if s
i
(A
i
k
) such that a
i
A
i
k
, u
i
(s
i
, a
i
)
u
i
(a
i
, a
i
) and b
i
A
i
k
such that u
i
(s
i
, b
i
) > u
i
(a
i
, b
i
) (elimination rule).
Example 2.6. Consider the following game:
L R
T 1, 1 0, 0
M 1, 1 2, 1
B 0, 0 2, 1
Figure 14: A game with weakly dominated actions
M weakly dominates B, so we can eliminate B. Then L weakly dominates R, so we can eliminate R
as well. This gives us the complete IEWDA A
T
= T, M L. However, there is a dierent complete
IEWDA. Looking at the original set of action proles again, M also weakly dominates T. After eliminating
T, R dominates L. Thus we have A

T
= M, B R. A
T
and A

T
are both complete but they are clearly
dierent.
This leads us to the following conclusion: For IESDA, the set of actions surviving any complete elimina-
tion is unique, i.e., the process of elimination is unimportant. But for IEWDA, the set of actions surviving
any complete elimination is NOT unique. Thus the process of elimination matters.
2.4 Nash Equilibria
Recall that if we have a mixed strategy s S, then the induced probability on A
i
is Pr
s
i (A
i
), where
Pr
s
i (a
i
) =

j=i
s
j
(a
j
). A few denitions:
Denition 2.12. Player is best response correspondence BR
i
: S S
i
is dened by
BR
i
(s) = s
i
S
i
: a
i
A
i
, u
i
(s
i
, Pr
s
i ) u
i
(a
i
, Pr
s
i ).
Denition 2.13. Player is pure best response correspondence PBR
i
: S A
i
is dened by
PBR
i
(s) = a
i
A
i
: b
i
A
i
, u
i
(a
i
, Pr
s
i ) u
i
(b
i
, Pr
s
i ).
Note that BR
i
S
i
while PBR
i
A
i
. The next example illustrates the relationship between the two.
Example 2.7. Consider the following game:
Suppose player 1 plays T with probability p and player 2 plays L with probability q. Then
PBR
1
(q, 1 q) =
_

_
T q > 1/4
T, B q = 1/4
B q < 1/4
12
L R
T 3, 1 0, 0
B 0, 0 1, 3
Figure 15: A game that illustrates PBR vs. BR correspondences
and
BR
1
(q, 1 q) =
_

_
(1, 0) q > 1/4
(T, B) q = 1/4
(0, 1) q < 1/4
So BR
1
(q, 1q) contains convex combinations of elements in PBR
1
(q, 1q). When q = 1/4, PBR
1
(q, 1q) =
T, B, and BR
1
(q, 1 q) is the set of all convex combinations of those two actions.
The example shows that in general, BR
i
(s) is the set of all probabilities on A
i
giving full measure to
elements in PBR
i
(s).
Lemma 2.1. Let V be compact and let : U V be a closed-valued correspondence. Then has a
closed-graph if and only if it is u.h.c.
Proof. Suppose is u.h.c. Let U x
n
x and let V y
n
(x
n
), n such that y
n
y V . Since is
closed-valued and V is compact, is compact-valued. Then the sequence denition of upper hemicontinuity
applies. Then a subsequence y
n
k
such that y
n
k
z (x). Every subsequence of a convergent sequence
must converge to the same limit as the mother sequence, so y = z. Then y (x), i.e., has a closed graph.
Now suppose that has a closed graph. Let U x
n
x and let V y
n
(x
n
), n. Since V is
compact, a subsequence y
n
k
such that y
n
k
y V . Since has a closed graph, y (x). Thus is
u.h.c.
Theorem 2.11. i I, BR
i
(s) is nonempty-valued, closed-valued, convex-valued and u.h.c.
Proof. Nonempty-valued: Fix s
i
S
i
. We can think of BR
i
(s) as argmax
s
i
S
i u
i
(s
i
, s
i
). Since S
i
=
(A
i
) is compact by denition, continuity of u
i
(, s
i
) implies that argmax
s
i
S
i u(s
i
, s
i
) ,= . Thus BR
i
(s) ,=
. Since s
i
was arbitrary, BR
i
is nonempty-valued.
Closed-valued: Let s S and let t
i
n
BR
i
(s) such that t
i
n
t
i
. Then u
i
(t
i
n
, s
i
) u
i
(r
i
, s
i
), r
i
S
i
.
By continuity of u
i
, u
i
(t
i
, s
i
) u
i
(r
i
, s
i
), so t
i
BR
i
(s). Thus BR
i
(s) is closed. Since s was arbitrary,
BR
i
is closed-valued.
Convex-valued: Let s S and let t
i
, r
i
BR
i
(s). Then it must be that u
i
(t
i
, s
i
) = u
i
(r
i
, s
i
)
u
i
(q
i
, s
i
), q
i
S
i
. By denition of u
i
(, s
i
), we have

a
i
A
i
t
i
(a
i
)u
i
(a
i
, s
i
) =

a
i
A
i
r
i
(a
i
)u
i
(a
i
, s
i
)

a
i
A
i
q
i
(a
i
)u
i
(a
i
, s
i
), q
i
S
i
.
Let [0, 1]. Then q
i
S
i
,

a
i
A
i
q
i
(a
i
)u
i
(a
i
, s
i
)

a
i
A
i
t
i
(a
i
)u
i
(a
i
, s
i
) +

a
i
A
i
(1 )r
i
(a
i
)u
i
(a
i
, s
i
)
=

a
i
A
i
[t
i
(a
i
) + (1 )r
i
(a
i
)]u
i
(a
i
, s
i
)
= u
i
(t
i
+ (1 )r
i
, s
i
).
Thus t
i
+ (1 )r
i
BR
i
(s) so BR
i
(s) is convex. Since s was arbitrary, BR
i
is convex-valued.
u.h.c.: Since S
i
(the range of BR
i
) is compact and BR
i
is closed-valued, it suces to show that BR
i
has a
closed graph. Let s
n
S such that s
n
s and let t
i
n
S
i
such that t
i
n
BR
i
(s
n
) and t
i
t
i
S
i
. We want
to show that t
i
BR
i
(s). Equivalently, we want to show that given u
i
(t
i
n
, s
i
n
) u
i
(r
i
, s
i
n
) r
i
S
i
, n,
the following inequality holds:
u
i
(t
i
, s
i
) u
i
(r
i
, s
i
), r
i
S
i
.
13
Let r
i
S
i
. Since s
n
s, s
i
n
s
i
. We can think of r
i
as a constant sequence, i.e., r
i
r
i
. Since t
i
n
t
i
,
continuity of u() implies that
u
i
(t
i
, s
i
) = lim
n
u
i
(t
i
n
, s
i
n
) lim
n
u
i
(r
i
, s
i
n
) = u
i
(r
i
, s
i
).
Thus t
i
BR
i
(s), so BR
i
is u.h.c.
It seems like we could also just use the maximum theorem to show that BR
i
is u.h.c. directly: u
i
() is
continuous and we can think of S
i
is a nonempty, compact-valued, continuous correspondence of s
i
(since
its nonempty, compact, and constant w.r.t. s
i
).
Denition 2.14. The best reply correspondence BR : S S is dened by
BR(s) =
iI
BR
i
(s).
Theorem 2.12. BR : S S is nonempty-valued, closed-valued, compact-valued and u.h.c.
Proof. Since BR(s) is the Cartesian product of BR
i
(s), i I, the result follows from the previous theorem.
With these results out of the way, we can move on to dening and proving the existence of Nash equilibria.
Denition 2.15. A Nash equilibrium for a NFG is a strategy prole s S such that s BR( s), i.e.,
s
i
BR
i
( s), i I.
Theorem 2.13. For every nite NFG, the set of Nash equilibria is nonempty and closed.
Proof. Clearly, S is nonempty, convex and compact. By the previous theorem, BR is nonempty-valued,
closed-valued, convex-valued and u.h.c. Then Kakutanis xed point theorem implies that BR has a xed
point s, i.e., s BR( s).
It remains to be shown that the set of Nash equilibria is closed. Let s
n
be a sequence in NE such that
s
n
s S. We want to show that s NE as well. Since s
n
NE, we have
u
i
(s
i
n
, s
i
n
) u
i
(t
i
, s
i
n
), i I, t
i
S
i
.
The desired conclusion follows from continuity of u
i
.
Example 2.8. Consider the following game:
L R
T 3, 1 0, 0
B 0, 0 1, 3
Figure 16: A game that illustrates the concept of Nash equilibrium
The set of Nash equilibria in this game is
_
_
(1, 0), (1, 0)
_
,
_
(0, 1), (0, 1)
_
,
_
(3/4, 1/4), (1/4, 3/4)
_
_
.
Further, we can see that ((1, 0), (1, 0)) = BR(((1, 0), (1, 0))) and ((0, 1), (0, 1)) = BR(((0, 1), (0, 1))).
However, BR
1
(1/4, 3/4) = (p, 1p) : p [0, 1]. Thus ((3/4, 1/4), (1/4, 3/4)) , = BR((3/4, 1/4), (1/4, 3/4)).
This example motivates the next denition.
Denition 2.16. s is a strict Nash equilibrium if s = BR( s).
In general, is a NE strict?
Example 2.9. Consider the following game.
Player 1 plays T with probability p and B with probability 1 p. Player 2 plays L with probability q
and R with probability 1 q. We can see that both ((1, 0), (1, 0)) and ((0, 1), (0, 1)) are Nash equilibria, but
only the former is strict.
14
L R
T 1, 1 0, 0
B 0, 0 0, 0
Figure 17: A game with strict and non-strict equilibria
Suppose s is a NE in an unspecied NFG. Then if there is a player i such that for some a
i
, b
i
A
i
with a
i
,= b
i
, s
i
(a
i
) > 0 and s
i
(b
i
) > 0, then s ,= BR( s). Thus s is not strict. Thus NE involving mixed
strategies are not strict.
Theorem 2.14. Let u denote the vector of player utilities (u
1
, . . . , u
n
). Consider the set of Nash equilibria
as a correspondence that maps from the set of player utility vectors, i.e.,
u = (u
1
, . . . , u
n
) NE(u) = s S : u
i
(s
i
, s
i
) u
i
(t
i
, s
i
), i, t
i
S
i
.
Then NE(u) is u.h.c.
Proof. Since S is compact and NE(u) is closed for all u by theorem 2.13, by lemma 2.1 it suces to show
that NE(u) has a closed graph. Let u
n
be a sequence of player utility vectors that converges to some u, and
let s
n
NE(u
n
), n such that s
n
s S. We want to show that s NE( u), i.e.,
u
i
( s
i
, s
i
) u
i
(t
i
, s
i
), i, t
i
S
i
.
We have
u
i
n
(s
i
n
, s
i
n
) u
i
(t
i
, s
i
n
), n, i, t
i
S
i
.
By continuity of u
n
, u
i
n
(s
i
n
, s
i
n
) u
i
n
( s
i
, s
i
) and u
i
(t
i
, s
i
n
) u
i
n
(t
i
, s
i
) for all n. Since u
n
u,
u
i
n
( s
i
, s
i
) u
i
( s
i
, s
i
) and u
i
n
(t
i
, s
i
) u
i
(t
i
, s
i
). Thus we have the desired result.
Theorem 2.15. Let E =
iI
E
i
be the action set surviving complete IESDA and let s be a Nash equilibrium.
Then
_
a A :

iI
s
i
(a) > 0
_
= support(s) E.
Proof. By induction. Let E
k
denote the set of actions surviving IESDA after k steps. Since E
0
= A,
support(s) A = E
0
by denition. Now suppose thhat support(s) E
k
. We want to showthat support(s)
E
k+1
.
Suppose not. Then a support(s) such that a , E
k+1
, i.e., for some i I, a
i
is strictly dominated by
some mixed strategy t
i
(E
i
k
). Then u
i
(t
i
, a
i
) > u
i
(a
i
, a
i
), a
i
E
i
k
. This implies that

a
i
E
i
k
u
i
(t
i
, a
i
)s
i
(a
i
) >

a
i
E
i
k
u
i
(a
i
, a
i
)s
i
(a
i
).
Since support(s) E
k
, for all a
i
, E
i
k
, s
i
(a
i
) = 0. Then

a
i
A
i
u
i
(t
i
, a
i
)s
i
(a
i
) >

a
i
A
i
u
i
(a
i
, a
i
)s
i
(a
i
)
or u
i
(t
i
, s
i
) > u
i
(a
i
, s
i
). This implies that a
i
, PBR
i
(s). This contradicts the fact that s NE, so
support(s) E
k+1
. The induction implies that support(s) E.
2.5 Correlated Equilibrium
Let ([a
i
) be player is posterior on A
i
given a
i
(given by Bayes rule) and (a
i
) is the probability that a
i
is recommended to player i.
15
Denition 2.17. A correlated strategy is a correlated equilibrium if i I, a
i
A
i
with (a
i
) > 0
and b
i
A
i
,

a
i
A
i
u
i
(a
i
, a
i
)(a
i
[a
i
)

a
i
A
i
u
i
(b
i
, a
i
)(a
i
[a
i
).
We can think of a correlated equilibrium as a correlated strategy such that if a A is drawn at random
with probability (a) and a
i
is recommended to player i, then player i (who knows ) might as well play a
i
if all the other players do the same (i.e., each other player j plays a
j
).
Note that
(a
i
[a
i
) =
(a
i
, a
i
)
(a
i
)
.
Then is a correlated equilibrium if i I, a
i
A
i
with (a
i
) > 0 and b
i
A
i
,

a
i
A
i
u
i
(a
i
, a
i
)
(a
i
, a
i
)
(a
i
)

a
i
A
i
u
i
(b
i
, a
i
)
(a
i
, a
i
)
(a
i
)
.
Multiplying both sides by (a
i
) and rearranging we see that is a correlated equilibrium if i I, a
i
A
i
with (a
i
) > 0 and b
i
A
i
,

a
i
A
i
[u
i
(a
i
, a
i
) u
i
(b
i
, a
i
)](a
i
, a
i
) 0.
Theorem 2.16. The set of correlated equilibria is nonempty, convex and closed.
Proof. Every Nash equilibrium is a correlated equilibrium, so by theorem 2.13, the set of CE is nonempty.
It is convex and closed because it is the set of solutions to a system of linear inequalities.
2.6 Zero-sum games (2 players)
Denition 2.18. A two-player zero-sum game is a NFG with u
1
= u
2
.
Theorem 2.17 (Min-max theorem). In a two-player, nite zero-sum game,
min
s
2
max
s
1
u
1
(s
1
, s
2
) = max
s
1
min
s
2
u
1
(s
1
, s
2
).
Proof. Proving the LHS is greater than or equal to the RHS is pretty trivial. Clearly,
max
s
1
u
1
(s
1
, s
2
) max
s
1
min
s
2
u
1
(s
1
, s
2
), s
2
.
Thus
min
s
2
max
s
1
u
1
(s
1
, s
2
) max
s
1
min
s
2
u
1
(s
1
, s
2
.
Note that this actually holds in any game, not just zero-sum games.
For the other direction, note that
max
s
1
min
s
2
u
1
(s
1
, s
2
) min
s
2
u
1
(s
1
, s
2
), s
1
.
In particular,
max
s
1
min
s
2
u
1
(s
1
, s
2
) min
s
2
u
1
( s
1
, s
2
),
where s is a Nash equilibrium, which we know exists by theorem 2.13. Then
min
s
2
u
1
( s
1
, s
2
) = u
1
( s
1
, argmin
s
2
u
1
( s
1
, s
2
))
= u
1
( s
1
, argmax
s
2
u
2
( s
1
, s
2
)) (by zero-sum property)
= u
1
( s
1
, s
2
) (since s is a Nash equilibrium)
= max
s
1
u
1
(s
1
, s
2
) (also since s is a Nash equilibrium)
min
s
2
max
s
1
u
1
(s
1
, s
2
)
16
Thus
max
s
1
min
s
2
u
1
(s
1
, s
2
) min
s
2
max
s
1
u
1
(s
1
, s
2
).
This proves the theorem.
2.7 Perfect Equilibrium
This section concerns a renement of the Nash equilibrium concept. We will start with an example to
motivate the discussion.
Example 2.10. In the game in gure 18 there are two Nash equilibria: (T, L) and (B, R). Note that both
B and R are weakly dominated strategies.
L R
T 1, 1 0, 0
B 0, 0 0, 0
Figure 18: A game with perfect and imperfect equilibria
Lets look more closely at the equilibrium (B, R). If player 2 plays R 100% of the time, player 1 is
indierent between T and B. But for any > 0 chance that player 2 plays L, player 1 will choose T for sure
since it gives higher expected utility. In other words, if player 1 has any doubt at all that player 2 might not
play R, player 1 will strictly prefer T. The same concept applies to player 2s decisionamking. Thus if both
players anticipate small mistakes, the equilibrium will be (close to) (T, L). This leads us to our next few
denitions.
Denition 2.19. The vector = (
1
(a
i
)
a
i
A
i )
iI
) is a perturbation vector if
i
(a
i
) > 0, i I, a
i
A
i
and

a
i
A
i

i
(a
i
) < 1, i I.
Denition 2.20. Given a NFG G and a perturbation vector , the perturbed game G

has the same

players utilities and action sets, but the players mixed strategies are constrained to the sets
S
i

= s
i
S
i
: s
i
(a
i
)
i
(a
i
), a
i
A
i
.
The constrained set of mixed strategy proles is S

=
iI
S
i

.
Denition 2.21. The set of Nash equilibria in the perturbed game G

is
NE(G

) = s S

: u
i
( s
i
, s
i
) u
i
(t
i
, s
i
), i I, t
i
S
i

.
Denition 2.22. A mixed strategy prole s is a perfect equilibrium if there exists a sequence of pertur-
bation vectors
n
such that
i
(a
i
) 0, i I, a
i
A
i
, and there exists a sequence s
n
NE(G
n
) such
that s
n
s. In other words, a perfect equilibrium is the limit of equilibria in some sequence of perturbed
games.
In the example above, note that ((0, 1), (0, 1)) is not a perfect equilibrium because in any constrained
game, players 1 and 2 will always play T and L respectively with the maximum allowed probabilities. Thus
no sequence of perturbed game equilibria converges to ((0, 1), (0, 1)).
Theorem 2.18. For every perturbation vector , NE(G

) ,= .
Proof. Clearly, S

is nonempty, compact and convex. The constrained best response correspondence is

BR
i

( s) = s
i
S
i

: u
i
(s
i
, s
i
) u
i
(t
i
, s
i
), t
i
S
i

.
We just need to verify that BR
i

(s) is nonempty, closed, convex and u.h.c. for all s S

. We can use the

exact same arguments as in the proof of theorem 2.12 to do this (basically hinges on continuity of u
i
). Then
BR

= BR
i

also has those properties, so Kakutani implies existence of a xed point. This is basically
exactly the same as proving existence of a Nash equilibrium in an unconstrained game.
17
Theorem 2.19. The set of perfect equilibria is nonempty.
Proof. For any sequence of perturbation vectors
n
0, we can use the previous theorem to get s
n

NE(G
n
). Then s
n
is a sequence in S. And since S is compact, there exists a convergent subsequence
s
n
k
s S. Then s is a perfect equilibrium by denition.
Theorem 2.20. A perfect equilibrium is a Nash equilibrium.
Proof. Let s be a perfect equilibrium. By denition, there exists a sequence of perturbation vectors
n
0
and a sequence s
n
NE(G
n
) such that s
n
s. We want to show that s NE(G), i.e.,
u
i
( s
i
, s
i
) u
i
(t
i
, s
i
), i I, t
i
S
i
.
We have
u
i
( s
i
n
, s
i
n
) u
i
(t
i
, s
i
n
), i I, t
i
S
i
n
.
For any t
i
S
i
we can nd a sequence t
i
n
S
i
n
such that t
i
n
t
i
. Then we have
u
i
( s
i
n
, s
i
n
) u
i
(t
i
n
, s
i
n
)
so the conclusion follows from continuity of u
i
.
There are some more theorems in this section, but before we state and prove them, it is helpful to prove
a theorem about equivalent denitions of perfect equilibrium. We need another denition rst.
Denition 2.23. Let > 0. A strategy prole s is -perfect if
(i) It is completely mixed, i.e., s
i
(a
i
) > 0, i I, a
i
A
i
.
(i) i I, a
i
j
A
i
, if a
i
k
A
i
such that u
i
(a
i
j
, s
i
) < u
i
(a
i
k
, s
i
), then s
i
(a
i
j
) .
Theorem 2.21. The following three statements are equivalent:
(A) s is a perfect equilibrium (as in denition 2.22).
(B) There exists a sequence of completely mixed strategy proles s
n
s such that s
i
is a best response to
s
i
n
for all i I and all n suciently large.
(C) There exists a sequence
n
0 such that
n
> 0 and a corresponding sequence of
n
-perfect strategies
s
n
such that s
n
s.
Proof. We will show that A C B A. First, suppose that s is a perfect equilibrium. Then there exists
a sequence of perturbation vectors
n
0 and a corresponding sequence of perturbed-game Nash equilibria
s
n
NE(G
n
) such that s
n
s. Since I and A are nite, we can dene
n
= max
iI,a
i
A
i
i
n
(a
i
). By
construction, s
i
n
BR
i
n
( s
n
), so any pure strategy a
i
that is not a best response to s
n
will be played with
probability
i
n
(a
i
). Thus each s
n
is
n
-perfect, so A C.
Now suppose that

t satises C. Then
n
0 and a corresponding sequence

t
n
of
n
-perfect strategies
such that

t
n


t. Since I and A are nite, we can nd d > 0 such that for all a
i
support(

t
i
),

t
i
n
(a
i
) d.
So for n large enough, d >
n
, which implies that any a
i
support(

t
i
) is a best response to

t
i
n
. This in turn
implies that

t
i
is a best response to

t
i
n
. Thus C B.
Finally, suppose r satises B. Then a sequence r
n
of completely mixed strategies such that r
n
r
and r
i
is a best response to r
i
n
for all i I and all n suciently large. We want to show that r is a perfect
equilibrium, i.e., a sequence of perturbation vectors
n
0 and a sequence r
n
NE(G
n
) such that
r
n
r. Dene
n
by

i
n
(a
i
) =
_
1/n a
i
support( r
i
)
r
i
n
(b
i
) b
i
, support( r
i
)
Then
i
n
(a
i
) > 0, i I, a
i
A
i
, n, and for n suciently large,

a
i
A
i
i
n
(a
i
) < 1, i I, and most
importantly,
i
n
(a
i
) 0, i I, a
i
A
i
. We then have r
i
n
(a
i
) r
i
(a
i
) > 0, a
i
support( r
i
) and
r
i
n
(b
i
) =
i
n
(b
i
), b
i
, support( r
i
). By assumption, r
i
is a best response to r
i
n
for all i I and all n
suciently large. Then for all i I, r
i
n
BR
i
n
( r
i
n
), the
n
-constrained best response set. Then we have
r
i
n
NE(G
n
) for n suciently large. Therefore B A.
18
Theorem 2.22. If s is perfect, then s is undominated.
Proof. Suppose not. Then for some i I, s
i
S
i
such that s
i
is weakly dominated by s
i
. Then
u
i
( s
i
, a
i
) u
i
(s
i
, a
i
), a
i
A
i
and b
i
A
i
such that
u
i
( s
i
, b
i
) < u
i
(s
i
, b
i
).
Since s is a perfect equilibrium, condition (B) in theorem 2.21 implies that there exists a sequence s
n
of fully
mixed strategies such that s
n
s and for n suciently large, s
i
BR
i
(s
n
) for all i I. In particular, this
holds for the player i who plays a weakly dominated strategy. This means that for n suciently large,
u
i
( s
i
, s
i
n
) u
i
(t
i
, s
i
n
), t
i
S
i
.
Thus for n suciently large,
u
i
( s
i
, s
i
n
) u
i
(s
i
, s
i
n
)
where s
i
is the strategy that weakly dominates s
i
. But since s
n
are fully mixed, Pr
s
i
n
(b
i
) > 0 for b
i
dened above, so we should have
u
i
( s
i
, s
i
n
) =

a
i
A
i
Pr
s
i
n
(a
i
)u
i
( s
i
, a
i
) <

a
i
A
i
Pr
s
i
n
(a
i
)u
i
(s
i
, a
i
) = u
i
(s
i
, s
i
n
).
Therefore we have a contradiction.
Lemma 2.2. Let X and Y be nonempty, compact, convex subsets of R
k
. Then x

X and y

Y such
that
x

y x

x y

, x X, y Y.
Proof. Note that X Y is also nonempty, compact and convex. Dene the correspondences X Y X
and : X Y Y as
(x, y) = argmax
xX
(x y)
and
(x, y) = argmax
yY
(x y).
By the maximum theorem, both and are nonempty, convex-valued, compact-valued and u.h.c. Dene
the correspondence : X Y X Y as
(x, y) = ((x, y), (x, y)).
Then is also nonempty, convex-valued, compact-valued and u.h.c. By Kakutanis xed point theorem,
(x

, y

) X Y such that (x

, y

) (x

, y

). Then x

(x

, y

) and y

(x

, y

), so it must be that
x

y x

x y

, x X, y Y.
Lemma 2.3. Let s
i
be an undominated mixed strategy in a two-player game with nite action sets. Then
s
i
is a best response to a completely mixed strategy.
Proof. Dene the set U as
U =
_
(u
i
(s
i
, a
i
1
), . . . , u
i
(s
i
, a
i
#A
i
)) : s
i
S
i
_
.
Let u

U correspond to a mixed strategy of player i that is undominated. We want to show that there
exists a p

(A
i
) such that p

0 and p

u, u U. Fix > 0 and dene the set P() as

P() =
_
_
_
p R
#A
i
: p
k
k = 1, . . . , #A
i
,
#A
i

k=1
p
k
= 1
_
_
_
.
19
Note that both U and P() are compact and convex subsets of R
#A
i
. Then by lemma 2.2, p() P()
and u() U such that
p() u p() u() p u(), p P(), u U.
WLOG, let u

= 0. Construct a sequence
n
0 so that u(
n
) u U. Since u

= 0, p(
n
) u

= 0.
Then we have
0 p(
n
) u(
n
) p u(
n
), n, p P(
n
).
By continuity of the dot product,
0 p(0) u p u, p P(0).
In other words, p u

p u, p P(0). Then u 0 = u

. Since u

is undominated, it must be that u = u

.
Therefore u(
n
) u

.
Let B U denote the set of elements of u for which there exists a strictly positive supporting hyperplane.
Since p(
n
) u p(
n
) u(
n
) and u(
n
) u

, we have u

B (the closure of B). Since U is the convex hull
of a nite number of vectors, B is closed. Then u

B, so p

0 such that p

u, u U. In
other words, p

is a completely mixed strategy to which u

is a best response.
Theorem 2.23. In a two-player, nite NFG, a nash equilibrium s is undominated if and only if it is a
perfect equilibrium.
Proof. The last two lemmas makes it easy to prove the desired result. Let s be an undominated Nash
equilibrium in a two-player game. By lemma 2.3, we know that s
i
is a best response to a completely mixed
strategy of player i. Let s
i
denote this completely mixed strategy, i.e., s
i
BR
i
( s
i
). We also know that
since s is a Nash equilibrium, s
i
BR
i
( s
i
). Then for any > 0, (1) s
i
+ s
i
is also a completely mixed
strategy, and s
i
BR
i
((1 ) s
i
+ s
i
). Let
n
0. Then for each i 1, 2, (1
n
) s
i
+
n
s
i
s
i
,
and s
i
BR
i
((1
n
) s
i
+
n
s
i
), n. By theorem 2.21, s is a perfect equilibrium.
Note that this equivalence does not hold with more than two players. The following example serves as
a counterexample.
Example 2.11. Consider the following three-player game:
Player 1
Player 2
L R
T 1, 1, 1 1, 0, 1
B 1, 1, 1 0, 0, 1
(Player 3 plays U)
Player 1
Player 2
L R
T 1, 1, 0 0, 0, 0
B 0, 1, 0 1, 0, 0
(Player 3 plays D)
Figure 19: A three-player game with an undominated Nash equilibrium that is not a perfect equilibrium
Player 1s action set is A
1
= T, B, player 2s action set is A
2
= L, R, and player 3s action set is
A
3
= U, D. Since the maximum payo for each player is 1, (B, L, U) is a Nash equilibrium in this game.
Note that L weakly dominates R, U weakly dominates D, and u
1
(B, R, D) > u
1
(T, R, D). Thus (B, L, U) is
also undominated
Note that player 2 always gets a payo of 1 from L and 0 from R regardless of what the other players
choose. Then in any perturbed game G

, he will always choose to assign the minimum possible probability

to R. Similarly, since player 3 always gets 1 from U and 0 from D, he will always assign the minimum
possible probabilty to D. Let s
1

= ((1
2
R
),
2
R
), (1
3
D
,
3
D
)) denote the mixed strategy that player 1s
rivals will inevitably choose in any perturbed game. Then for any vector of perturbations with
1
B
,
2
R
and

3
D
all suciently small (which will be satised by any sequence of vectors
n
0),
u
1
((1
1
B
,
1
B
), s
1

) > u
1
((
1
T
, 1
1
T
)

, s
1

).
Thus ((
1
T
, 1
1
T
), s
1

) , NE(G

) for any with

1
B
,
2
R
and
3
D
all suciently small. This implies that
(B, L, U) is not the limit of any sequence of perturbed-game equilibria, i.e., (B, L, U) is not a perfect
equilibrium.
20
3 Extensive Form Games
Example 3.1. Before we begin this section, it is helpful to show a few examples of extensive form games.
1
z
1
L
2
R
z
2

z
3
r
Figure 20: A simple extensive game
1
2
A
2, 1

4, 0
r
2
B
1, 3

1, 0
r
2
C
1, 0

2, 1
r
0, 1
D
Figure 21: A more complicated extensive form game
Throughout this section, the label above a node contains the nodes identier and/or the player who owns
it. Dotted lines around nodes denote an information set, i.e., thw player who owns the nodes surrounded by
a dotted line cannot tell which of the nodes he is at.
3.1 Preliminaries
Denition 3.1. An extensive form game is made up of:
Players indexed by I = 1, . . . , n
Nature (sometimes given the zero index)
X, the set of nodes
Z, the set of nal nodes
o, the initial node
Player utilities u
i
: Z R, i I
, a partial order over X
x y means x comes before y.
is reexive, asymmetric and transitive.
For all x X, o x.
For all x X, the set of predecessor nodes p(x) y : y x is linearly ordered by .
For all x X, the set of immediate predecessors is
P(x) z p(x) : y such that z ,= y, z ,= x, z y x.
21
For all x X, the set of immediate successor nodes is
S(x) z : z x, y such that z y x.
Z = x X : S(x) = .
Before we can get into the meath of this section, we need to specify more clearly the players that can
act at each node, the information they know, and the actions they can take.
Denition 3.2. The players partition is (P
1
, . . . , P
n
) where for all i ,= j, P
i
P
j
= and
iI
P
i
=
X o, Z.
Denition 3.3. For all i I, the information partition U
i
= u, v, w, . . . is a partition of P
i
, i.e.,
u v v . . . = P
i
and u v = , u ,= v.
3.2 Properties of extensive form games
Denition 3.4. An extensive form game is linear if for all i I, all u U
i
and all z Z, #p(z) u 1.
Example 3.2. The game in gure 22 is nonlinear because the information set x, w is crossed twice by
the path to z
2
(and z
3
), i.e., #p(z
2
) x, w = 2. Compare this with the game in gure 20 (a simple
extensive game), which is linear. In general, we will exclude nonlinear games from our analysis, although
they are occasionally useful as examples of other concepts.
1
z
1
L
1
R
z
2

z
3
r
u
Figure 22: A nonlinear extensive form game
Denition 3.5. An extensive form game is of perfect recall for player i if u, v U
i
, y, w v and
c C
u
such that y follows c but w does not follow c.
Example 3.3. Figure 23 shows a game that it is not of perfect recall. We will generally want to exlude
these games as well.
1
1
L
z
1

z
2
r
1
R
z
3

z
4
r
u
Figure 23: An extensive form game that is not of perfect recall
Example 3.4. We will also exclude games in which for some player i, u U
i
and x, y u such that
#S(x) ,= #S(y). The game in gure 24 illustrates this concept. One of the nodes in player 2s information
set u, v has 2 successor nodes while the other has three. This kind of game really doesnt make much
sense.
22
1
2
L
z
1

z
2
r
2
R
z
3

z
4
m
z
5
r
u
Figure 24: Player 2 has a dierent number of possible actions at each node in his information set
From the previous example, we can see that we need to formally dene the actions that a player can
take at any information set in a way that is consistent with multiple-node information sets.
Denition 3.6. Given an information set u, C
u
is the set of choices allowed by the player who owns u. C
u
is a partition of
xu
S(x) into sets such that each set contains one and only one element of S(x) for each
x u.
3.3 Strategies
As in normal form games, extensive form games have pure (deterministic) and mixed strategies.
Denition 3.7. The set of pure strategies for player i is S
i
= s
i
= (s
i
u
)
uU
i : s
i
u
C
u
.
Denition 3.8. The set of mixed strategies for player i is
i
= (S
i
).
In extensive form games, a mixed strategy is a probability measure on S
i
. A player using a mixed
strategy randomizes once before playing the game, so he will choose the same c C
u
each time he gets to a
node in information set u. But what if we want to randomize over C
u
at each node in u?
Denition 3.9. The set of behavioral strategies for player i is B
i
=
uU
i (C
u
).
Example 3.5. Figure 25 shows another nonlinear game. In this game, player 1 has one information set
u = x, y. Then the set of pure strategies is S
1
= C
u
= L, R. The set of mixed strategies is

1
= (S
1
) = (p, 1 p) : p [0, 1].
What is the set in (Z) (probability distributions on nal nodes) we can achieve with S
1
? Clearly, L induces
the distribution (1, 0, 0) and R induces (0, 0, 1), so we can achieve (1, 0, 0), (0, 0, 1) (Z) with S
1
. What
about with
1
? For each (p, 1 p)
1
, p = Pr(L). Thus
1
induces (p, 0, 1 p) : p [0, 1] (Z).
What about behavioral strategies? Each b
1
B
1
is an element of (C
u
). But while B
1
may appear to
be the same thing as
1
, B
1
induces the distribution (q, q(1 q), (1 q)
2
) : q [0, 1] (Z).
1
2, 2
L
1
R
3, 1

0, 0
r
u
Figure 25: Another nonlinear extensive form game
There is one more kind of strategy in an extensive form game.
23
Denition 3.10. The set of general strategies for player i is
i
= (B
i
).
say something about relationships between the various sets of strategies
Denition 3.11. Two strategies of player i are equivalent if they induce the same probability distribution
on Z, for any general strategy of the other players.
Theorem 3.1 (Kuhns Theorem). For games of perfect recall for player i, for any behavioral strategy b
i
B
i
,
there is an equivalent mixed strategy
i

i
, and vice versa.
Example 3.6. Consider the game shown in gure 26. We can write any mixed strategy as
= ((L, ), (L, r), (R, ), (R, r)).
Similarly, we can write any behavioral strategy b as
b = (b
u
(L) = q
1
, b
v
() = q
2
).
Given a behavioral strategy b, we need to nd a mixed strategy that induces the same probabilities on
nal nodes. So this has to have
(L, ) +(L, r) = q
1
(R, )
(R, ) +(R, r)
= q
2
In this example, if we are given a mixed strategy, the same equations give the equivalent behavioral strategy.
1
z
1
L
2
R
z
2

z
3
r
Figure 26: Illustration of Kuhns theorem
Theorem 3.2. Perfect recall linear.
Proof. Let G be an extensive form game of perfect recall for every i I. Suppose G is not linear. Then for
some i I, there exist nodes x
1
, x
2
and an information set u U
i
such that x
1
x
2
and x
1
, x
2
u.
nish this!
Example 3.7. Note that linear not perfect recall. Consider the game shown in gure 27
Theorem 3.3 (Isbells theorem). In any linear extensive form game, for every behavioral strategy there is
an equivalent mixed strategy.
Proof. Let z Z be a nal node. Let h(z) denote the history of choices that leads from the initial node o to
z. In other words, for any strategy prole (pure, mixed, behavioral or general), h(z) represents the sequence
of choices that must be realized in order to get to z. Let b
i
B
i
be a behavioral strategy for player i. Given
a general strategy of player is opponents
i
, the probability induced on nal node z Z is
Pr
(b
i
,
i
)
(z) =
_

ch(z)Cu
uU
i
b
i
u
(c)
_

_
_
B
i
_

ch(z)Cu
uU
i
b
i
u
(c)
_

_
d
i
(b
i
).
24
1
1
L
z
1

z
2
r
2 R
1
A
z
3

z
4
r
z
5
B
u
Figure 27: A linear EFG that is not of perfect recall
Since the game is linear, for each nal node z there exists a pure strategy prole ( s
i
, s
i
) S
i
S
i
that reaches z. In other words, h(z) = ( s
i
, s
i
). Then
Pr
(b
i
,
i
)
(z) =
_
_

s
i
u
,uU
i
b
i
u
( s
i
u
)
_
_
_
B
i
_
_

s
i
u
,uU
i
b
i
u
( s
i
u
)
_
_
d
i
(b
i
).
This implication of linearity also lets us construct a mixed strategy
i
dened as

i
(s
i
) =
i
((s
i
u
)
uU
i ) =

uU
i
b
i
u
(s
i
u
), s
i
S
i
.
Then the probability on z induced by
i
and
i
is
Pr
(
i
,
i
)
(z) =
i
( s
i
)
_
B
i
_
_

s
i
u
,uU
i
b
i
u
( s
i
u
)
_
_
d
i
(b
i
).
By denition of
i
,

i
( s
i
) =
i
(( s
i
u
)
uU
i ) =

uU
i
b
i
u
( s
i
u
),
so we have shown that
Pr
(b
i
,
i
)
(z) = Pr
(
i
,
i
)
(z).
Thus b
i
and
i
induce the same probability on nal nodes given an arbitrary general strategy of the opponents,
i.e.,
i
is equivalent to b
i
.
Theorem 3.4. In a game which is linear for player i, every general strategy of i is equivalent to a mixed
strategy of i.
Proof. coming soon!
3.4 Nash equilibria and renements
Denition 3.12. The normal form of an EFG is a normal form game with the set same of players where
for all i I, player is action set is S
i
(his set of pure EFG strategies) and his utility u
i
(s) = u
i
(s
1
, . . . , s
n
)
for the strategy prole s is dened as the expected utility at nal nodes induced by s. The mixed extension
is dened as usual (
i
is the set of mixed strategies for player i).
Example 3.8. Figure 28 contains a simple extensive form game.
The normal form game that represents the game in gure 28 is shown in gure 29
25
1
2, 2
L
2
R
3, 1

0, 0
r
Figure 28: An extensive form game...
r
L 2, 2 2, 2
R 3, 1 0, 0
Figure 29: ... and its normal form
Denition 3.13. A Nash equilibrium of an extensive form game is a Nash equilibrium of the mixed extension
of its normal form.
Theorem 3.5. The set of Nash equilibrium in any nite extensive form game is nonempty.
Proof. Obvious by theorem 2.13.
So we know that any nite extensive form game will have a Nash equilibrium in mixed strategies. But
what about behavioral strategies?
Denition 3.14. A Nash equilibrium in behavioral strategies is a behavioral strategy prole

b = (

b
1
, . . . ,

b
n
)
such that for all i I,

b
i
is a best response to

b
i
.
Theorem 3.6 (Corollary to Kuhns theorem). In any extensive form game of perfect recall for every player,
for Nash equilibrium in mixed strategies there is a NE in behavioral strategies that induces the same probability
on nal nodes (and hence the same payos), and vice versa. Therefore the set of Nash equilibrium in
behavioral strategies is nonempty in such a game.
Proof. Obvious by Kuhns theorem and theorem 3.5.
3.4.1 Games of perfect information
Denition 3.15. An extensive form game is of perfect information if for all i I and all u U
i
, u is a
singleton.
Denition 3.16. Given an extensive form game G and a node x X, a subgame is another extensive form
game in which x is the initial node, and the set of nodes is the subset of X that is connected to x by paths.
In particular, every player must know what the game is, so every player must know with certainty that the
game starts at node x.
In games of perfect information, there is a subgame that begins at every non-nal node. However, this
is not true in games of imperfect information because players may not know which of the nodes in their
information set they are at.
Example 3.9. show why this is the case
Denition 3.17. A subgame perfect equilibrium is a Nash equilibrium that is also a Nash equilibrium
in every subgame.
Theorem 3.7 (Zermelo). Every extensive form game of perfect information has an equilibrium in pure
strategies that is subgame perfect.
26
Sketch of proof. We can use backward induction to show that a subgame perfect pure strategy equilibrium
exists.
(1) For each nal node z, look at the set of immediate predecessors P(z).
(2) Each node (not information set) x in P(z) has a player who owns it (acts at that node). Choose the
action that gives this player the largest payo (which will be deterministic). Break ties arbitrarily.
(3) Replace each node in P(z) with a nal node having utility for each player equal to the utilities induces
by the action chosen in step 2.
(4) Repeat steps 1 - 3 until only one node is left.
The actions chosen in this process describe a pure strategy prole that is a subgame perfect equilibrium.
3.4.2 Games of imperfect information
Denition 3.18. Given a node x X, the subtree beginning at x is K
x
= y X : x y (the set of
nodes that follows x).
Given x X and subtree K
x
, we call
x
the restriction of on K
x
, i.e., x
x
y if and only if x y and
x, y K
x
. Similarly, we have the restricted player partition P
i
x
= P
i
K
x
, the restricted set of nal nodes
Z
x
= Z K
x
, and u
i
x
: Z
x
R, the restriction of u
i
on Z
x
. As mentioned above, for games of imperfect
information, not all subtrees are subgames.
Denition 3.19. A subtree K
x
forms a subgame with all of the resitrictions mentioned above if for every
player i and every information set u U
i
, (x
1
, x
2
u && x
1
K
x
) x
2
K
x
. In other words, for any
information set u with u K
x
,= , u K
x
.
Example 3.10. In the game shown in gure 30, neither of the two subtrees beginning at the second level
of the game tree are subgames. The only subgame is the whole game.
1
2
L
z
1

z
2
r
2
R
z
3

z
4
r
u
Figure 30: Not all subtrees are subgames
Denition 3.20. A minimal subgame is a subgame that has no subgames other than itself.
Subgame perfect equilibria are dened exactly the same for games of imperfect information as they are
above. In fact, subgame perfect equilibria exist in every game of imperfect information as well.
Theorem 3.8. In any extensive form game G of imperfect information, the set of subgame perfect equilibria
is nonempty.
Proof. The proof of this theorem is very similar to the proof in the perfect information case. The backward
induction goes as follows:
(1) Take all the minimal subgames G
x
of G.
27
(2) For each of these subgames, nd a Nash equilibrium behavior strategy prole (which we know exists
from the corollary to Kuhns theorem).
(3) Replace the original game with the game obtained by replacing the node x (the initial node of the
subgame G
x
) with the expected payo from the NE found in (3).
(4) Repeat this process until only one node remains.
Combining the subgame NE behavioral strategy proles from the process above gives us a behavioral
strategy prole for the whole game. It remains to be shown that this strategy prole is a NE of every
subgame. To do this we need to introduce some notation. Given a subgame G
y
with subgame G
x
, let

b
x
denote the NE strategy prole for G
x
obtained in (2) above. Let G(x, b
x
) denote the game obtained from
replacing G
x
with the expected payo from

b
x
as in (3). Let b
x
denote a behavioral strategy prole for all
the information sets in G
y
other than those contained in G
x
. Finally, let x
x
denote the NE strategy prole
on G(x, b
x
).
So we know

b
x
is a NE of G
x
and

b
x
is a NE of G(x,

b
x
). We want to show that

b (

b
x
,

b
x
) is a NE
on G
y
. In other words, we want to show that for every i I,

b
i
BR
i
y
(

b). Given an arbitrary strategy b

i
,
player is expected payo when the other players play

b
i
is

zZx
u
i
(z)Pr
(b
i
,

b
i
)(z)
+

zZy\Zx
u
i
(z)Pr
(b
i
,

b
i
)
(z).
This can be rewritten as
Pr
(b
i
,

b
i
)
(x)

zZx
u
i
(z)Pr
(b
i
,

b
i
)(z|x)
+

zZy\Zx
u
i
(z)Pr
(b
i
,

b
i
)
(z).
Note that even if Pr
(b
i
,

b
i
)
(x) is zero, Pr
(b
i
,

b
i
)
(z[x) is still well-dened. Since (

b
x
,

b
x
) is a NE in G
x
,
max
b
i
B
i

zZx
u
i
(z)Pr
(b
i
,

b
i
)
(z[x) =

zZx
u
i
(z)Pr
(

b
i
x
,

b
i
x
)
(z[x) u
i
(x).
Then since

b
x
is a NE in G(x),
Pr
(b
i
,

b
i
)
(x)

zZx
u
i
(z)Pr
(b
i
,

b
i
)(z|x)
+

zZy\Zx
u
i
(z)Pr
(b
i
,

b
i
)
(z)
Pr
(b
i
,

b
i
)
(x)u
i
(x) +

zZy\Zx
u
i
(z)Pr
(b
i
,

b
i
)
(z)
Thus player i will choose to play

b
i
x
in G
x
.
Let b
i
x be an arbitrary strategy in G(x,

b
x
). Given player is decision to play

b
i
x
in G(x), his expected
payo for an arbitrary strategy b
i
x
in G(x,

b
x
) is
Pr
(b
i
x
,

b
i
x
)
(x)u
i
(x) +

zZy\Zx
u
i
(z)Pr
(b
i
x
,

b
i
x
)
(z).
Note that we can replace Pr
(b
i
,

b
i
)
(x) with Pr
(b
i
x
,

b
i
x
)
(x) since only the partial strategy prole (b
i
x
,

b
i
x
)
is what determines the probability of getting to node x (and thus G(x)). Since

b
x
is a NE in G(x,

b
x
), for
all b
i
x
we have
Pr
(b
i
x
,

b
i
x
)
(x)u
i
(x) +

zZy\Zx
u
i
(z)Pr
(

b
i
x
,

b
i
x
)
(z) Pr
(b
i
x
,

b
i
x
)
(x)u
i
(x) +

zZy\Zx
u
i
(z)Pr
(

b
i
x
,

b
i
x
)
(z).
Thus player i will choose to play

b
i
x
in G(x, x). Therefore (

b
i
x
,

b
i
x
) BR
i
y
(

b), so we have shown that

(

b
x
,

b
x
) is a NE in G
y
.
28
3.4.3 Sequential equilibria in extensive form games
Consider the game in the following gure. If we require player 2 to state his beliefs at u of x vs. y, then the
choice r is dominated by .
1
2, 2
L
x
M
3, 1

0, 0
r
y 2
R
3, 1

0, 0
r
u
Figure 31: Player 2 should always choose when the game gets to his info set u.
Denition 3.21. A system of beliefs is a map : X [0, 1] such that for all i I and all u U
i
,

xu
(x) = 1.
Denition 3.22. An assessment is a pair (b, ), where b is a behavioral strategy prole and is a system
of beliefs.
Denition 3.23. For any > 0, the set of -fully mixed behavioral strategies for i is
B
i

= b
i

B
i
: u U
i
, c C
u
, b
i

(c) .
For every b

=
iI
B
i

(x[u) =
Pr
b
(x)
Pr
b
(u)
is well-dened and unique.
Denition 3.24. An assessment (b, ) is consistent if (b

) for 0 such that

1. b

b.
2.

.
3. For every > 0,

is induced by b

according to Bayes rule as above.

Denition 3.25. The behavioral strategy b
i
is a sequential best response of player i to (b, ) if for every
u U
i
,

xu
(x)P
(b
i
,b
i
)
x
(z)u
i
(z)

xu
(x)P
(b
i
,d
i
)
x
(z)u
i
(z)
for all d
i
B
i
, where P
b
x
(z) is the probability of reaching z if we start from x and everyone plays b.
Denition 3.26. A sequential equilibrium is an assessment (b, ) such that
1. (b, ) is consistent.
2. For every i I, b
i
is a sequential best response to (b, ).
Example 3.11. Consider the game in the next gure. The behavioral strategy b = ((1, 0, 0), (0, 1)) is
subgame perfect, but there is no system of beliefs such that b
2
= (0, 1) is a sequential best response to
(b, ). Therefore there exists no such that (b, is a sequential equilibrium.
29
1
2, 2
L
x
M
3, 1

0, 0
r
y 2
R
3, 1

0, 0
r
u
Figure 32: The behavioral strategy where player 1 plays L and player 2 plays r all the time is subgame
perfect, but it is not a sequential equilibrium.
30