ECS 122A: Design and Analysis of Algorithms

Discussion Notes – January 8, 2010

Samuel Johnson Winter 2010

1 Asymptotic Notation
In this section we review the definitions for certain asymptotic notations.
O(g(n)): A function f is O(g(n)) iff there exist positive constants c, and n0 such that

0 ≤ f (n) ≤ c · g(n)

for all n ≥ n0 .
o(g(n)): A function f is o(g(n)) iff for any positive constant c, there exist a positive constant n0
such that
0 ≤ f (n) < c · g(n)
for all n > n0 .
Ω(g(n)): A function f is Ω(g(n)) iff there exist positive constants c and n0 such that

0 ≤ c · g(n) ≤ f (n)

for all n ≥ n0 .
ω(g(n)): A function f is ω(g(n)) iff for any positive constant c, there exists a positive constant
c0 such that
0 ≤ c · g(n) < f (n)
for all n ≥ n0 .
Θ(g(n)): A function f is Θ(g(n)) iff there exist positive constants c1 , c2 , and n0 such that

0 ≤ c1 g(n) ≤ f (n) ≤ c2 g(n)

for all n ≥ n0 . That is, f is Θ(g(n)) iff f is both O(g(n)) and Ω(g(n)).

2 Substitution Method
The substitution method for solving analyzing recurrences is a three-step process:
1. Guess the form of the solution;
2. Verify the solution by using an inductive argument;
3. Solve for constants.
The remainder of this section illustrates the use of the substitution method.

1
Example 1
Suppose we are given the recurrence

T (n) = 4T (n/2) + n (1)

2
Let us guess that our solution to (1) is O(n ). To prove this inductively, lets use (as the
inductive hypothesis) T (n) ≤ cn2 − bn, and assume that it is true for T (n/2). That is, assume
that T (n/2) ≤ c(n/2)2 − b(n/2) is true.

T (n) ≤ 4(c(n/2)2 − b(n/2)) + n

= 4c(n/2)2 − 4b(n/2) + n
= cn2 − 2bn + n
= cn2 − n(2b − 1)
≤ cn2 − nΘ(1)

where the last inequality holds so long that b > 1/2.

Example 2
Suppose we are given the recurrence

T (n) = 2T (⌊n/2⌋) + n. (2)

1
We must first guess a solution for (2). Let us guess that T (n) = O(n lg n). The substitution
method now requires us to show that T (n) ≤ cn lg n for some constant c > 0.
The proof will be inductive. That is, we assume that T (⌊n/2⌋) ≤ c⌊n/2⌋ lg(⌊n/2⌋) holds, and
our goal is to use this to show that this yields our claim that T (n) ≤ cn lg n.

T (n) ≤ 2(c⌊n/2⌋ lg(⌊n/2⌋)) + n

≤ cn lg(n/2) + n
= cn lg n − cn lg 2 + n
= cn lg n − cn + n
≤ cn lg n.

Example 3
Suppose that we are given the recurrence

T (n) = T (⌊n/2⌋) + T (⌈n/2⌉) + 1. (3)

We guess that T (n) = O(n), and want to show T (n) ≤ cn:

T (n) ≤ c⌊n/2⌋ + c⌈n/2⌉ + 1

= cn + 1.
1 Here and in many other places, we use the shorthand notation lg n with the understanding that lg n ≡ log2 n.

2
However, this is incorrect since it does not imply T (n) ≤ cn for all values of c. Instead, we should
guess T (n) ≤ cn − b for some constant b ≥ 0, giving us:

T (n) ≤ (c⌊n/2⌋ − b) + (c⌈n/2⌉ − b) + 1

= cn − 2b + 1
≤ cn − b

where the final inequality holds if our constant b ≥ 1. The arithmetic for the last inequality
seems a bit off. However, what is important is that the form of the conclusion matches that
of the inductive hypothesis. Now, since the result of our induction matches our hypothesis, we
can infer our desired O(n) outcome since (asymptotically) the cn term dominates the (constant)
term b, yielding our result: T (n) = O(n).
For more details regarding this and the previous example, refer to chapter 4.1 in CLRS.2

Example 4
Show that

T (n) = T (⌈n/2⌉) + 1 (4)

is O(lg n). We guess that T (n) ≤ c lg(n − b) for some b > 0, and assume that T (n/2) ≤
c lg(⌈n/2⌉ − b) holds. Next is the induction:

T (n) ≤ c lg(⌈n/2⌉ − b) + 1
≤ c lg((n + 1 − 2b)/2) + 1
= c lg(n + 1 − 2b) − lg 2 + 1
= c lg(n + 1 − 2b)
≤ lg(n − Θ(1))

where it must be that b > 1/2.

2 Recurrences are discussed in chapter 4 of the second edition of CLRS. At this time, I am unsure if the same

chapter number carries over to the third edition.