Mathematical preliminaries

Exponentials
1
a
m n
2)(a ) = a mn

1)a 1 =

3)a m a n = a m+ n
n

x
4) Lt n 1 + = e x
n
Logarithms
x = logba is the exponent for a = bx.
Binary log: lg a = log2a
Natural log: ln a = logea
For all real a>0, b>0, c>0 and n
1)a = b logb a
2) log c (ab) = log c a + log c b
3) log b a n = n log b a
4) log b a =

log c a
= log c a * log b c
log c b

5) log b (1 / a ) = log b a
6) log c (a / b) = log c a log c b
7) log b a =

1
log a b

8)a logb c = c logb a

9)lg 2 a = (lg a) 2
10) lg lg a = lg(lg a )
11) (i) If b constant then logba strictly increasing as a increases.
(ii) If a constant then logba strictly deccreasing as b increases.
12) Base of logarithm is not an issue in asymptotic notation.i.e., Changing the base of
logarithm from one constant to another changes the value by a constant.
logba = k (logca) K is constant.
13) Exponentials with different bases differ by a exponential factor (not a constant factor).
Ex: 2n = (2/3)n*3n.

Monotonicity
f(n) is
(i) monotonically increasing if m n f(m) f(n).
(ii) monotonically decreasing if m n f(m) f(n).
(iii) strictly increasing if m < n f(m) < f(n).
(iv) strictly decreasing if m > n f(m) > f(n).
Sequences
Arithmetic sequence
a,a+d,a+2d,..,a+(n-1)d,..
nth term of the sequence tn= a+(n-1)d
Sum to first to n terms Sn=n/2[2a+(n-1)d]
Geometric sequence
a, ar,ar2, ar3,,arn-1,
Sum to first to n terms

Sn=
Sn=

Sum to infinite terms

( )

if r>1

( )

if r<1

S=

|r|<1

Constant Series: For integers a and b, a b,

b

1 = b a + 1
i =a

Linear Series (Arithmetic Series): For n 0,

n

i = 1 + 2 + + n =
i =1

Quadratic Series: For n 0,

n

= 12 + 2 2 + + n 2 =

i =1

n(n + 1)
2

n(n + 1)(2n + 1)
= ( n 2 )
6

Cubic Series: For n 0,

n 2 (n + 1) 2
i = 1 + 2 ++ n =

4
i =1
n

Geometric Series: For real x 1,

n

xk = 1+ x + x2 + + xn =
k =0

For |x| < 1,

x
k =0

1
1 x

x n +1 1
x 1

Linear-Geometric Series: For n 0, real c 1,

n

ic i = c + 2c 2 + + nc n =
i =1

(n + 1)c n +1 + nc n + 2 + c
(c 1) 2

Harmonic Series: nth harmonic number, nI+,

H n = 1+

1 1
1 n 1
+ + + = = ln(n) + O(1)
2 3
n k =1 k
1

n1

ni=1 1 dx + 1 = ln n + 1
i
x

Solution:

Other important formulas

n

= 1p + 2 p + + n p

i =1

1
n p +1 O(n p +1 )p 0
p +1

lg k n lg n
k =1

Factorials
n! = 1.2.3.4.n
n! nn, thus n!=O(nn)
Stirlings Approximation
n
n!= 2n
e

1
1 +
n

From Stirlings approximation, the followings hold:

n! = o(nn)
n! = (2n)
lg (n!) = (n lg n)

2. Asymptotic Notation
Asymptotic efficiency of algorithms is concerned with how the running time increases with the
size of the input in the limit, i.e. as the size of the input increases without bound.
Definition:
Asymptotic complexity is running time of an algorithm as a function of input size n for large n

An asymptotically more efficient algorithm is the best choice.

Written using Asymptotic Notation.

O-notation (Big-oh Notation) (Asymptotic Upper bound )

O (g (n)) = {f(n) : positive constants c and n0, such that
n n0, we have 0 f(n) cg(n) }
Intuitively: Set of all functions whose rate of growth is the same as or lower than that of g (n).

If f (n) O (g (n)), we write f (n) =O(g(n)).

Examples
1) 2n2 = O(n3)
2n2 cn3

2 cn

c = 1 and n0= 2

2) 1000n2+1000n = O(n2)
1000n2+1000n 1000n2+ n2 =1001n2
3) Examples of of functions inO (n2)
n2, n2+n, 5n2+6n, 1000n2+100n+10.
Also, n, n/100, n1.99999, n2/lg lg n

c=1001 and n0 = 1000

 -notation(Big-Omega Notation) (Asymptotic Lower bound)

( g(n)) = {f (n) : positive constants c and n0, such that
n n0, we have 0 cg(n) f(n) }
Intuitively: Set of all functions whose rate of growth is the same as or higher than that of g(n).

Examples
1) 5n2 = (n)
c, n0 such that: 0 cn 5n2

cn 5n2

c = 1 and n0 = 1

2) Examples of of functions in (n2)

n2, n2+n, 1000n2+1000n, 1000n2-1000n
Also, n3, n2.000001, n2lg lg n,2n
-notation(Theta-Notation) (Asymptotic tight bound)
(g (n)) = { f (n) : positive constants c1, c2, and n0, such that
n n0, we have 0 c1g(n) f(n) c2g(n) }

Technically, f(n) (g(n)). Older usage, f(n) = (g(n)).

Example:
1) n2/2 n/2 = (n2)
n2 - n n2 n 0 c2=
n2 - n n2 - n * n ( n 2 ) = n2

c1=

o-notation
o(g(n)) = {f(n): c > 0, n0 > 0 such that
n n0, we have 0 f(n) < cg(n)}.
g(n) is an upper bound for f(n) that is not asymptotically tight.
w notation
w(g(n)) = {f(n): c > 0, n0 > 0 such that
n n0, we have 0 cg(n) < f(n)}.
g(n) is a lower bound for f(n) that is not asymptotically tight.
Comparisons:
f (n) is asymptotically smaller than g(n) if f (n) =o(g(n)).
f (n) is asymptotically larger than g(n) if f (n) =(g(n)).
Comparison of Functions
f (n) = O(g(n)) a b
f (n) = (g(n)) a b
f (n) = (g(n)) a = b
f (n) = o(g(n)) a < b
f (n) = w (g(n)) a > b
Note: But, No trichotomy. i.e., we might not be able to compare functions.
Example: n1+sin n and n
PROPERTIES
1) f(n) = (g(n)) f(n) = (g(n)). Therefore, (g(n)) O (g (n)).
2) f(n) = (g(n)) f(n) = O(g(n)).Therefore,(g(n)) (g (n)).

3) Theorem :
For any two functions g(n) and f(n),
f(n) = (g(n)) iff f(n) = O(g(n)) and f(n) = (g(n)).
i.e., (g(n)) = O(g(n)) (g(n)

4) Transitivity (Same for O, , o, )

f(n) = (g(n)) and g(n) = (h(n)) f(n) = (h(n))
5) Reflexivity (Same for O,)
f(n) = (f(n))
6) Symmetry
f(n) = (g(n)) if and only if g(n) = (f(n))
7) Transpose symmetry
f(n) = O(g(n)) if and only if g(n) = (f(n))
f(n) = o(g(n)) if and only if g(n) = (f(n))
8)Limit Formula: Let

()
=
)

1. If A=0, then
f(n)=o(g(n))
or
f(n)=O(g(n)) and f(n) (g(n))
2 If A=, then
f(n)= (g(n))
or
f(n)= (g(n)) and f(n) (g(n))
3. If A0 and finite , then f(n) = (g(n))

9) If f(n)=a0+a1 n+..+am nm and am>0 then f(n)= (nm)

10) Multiplication by a constant does not change the asymptotics:
O(c f(n)) O(f(n))
(c f(n)) (f(n))
(c f(n)) (f(n))

11) Similar properties hold for and

(i) If T1(n) is O(f(n)), T2(n) is O(g(n))
then T1(n) * T2(n) is O(f(n)*g(n))
(ii) If T1(n) is O(f(n)), T2(n) is O(g(n))
then T1(n) + T2(n) is O(max(f(n),g(n)))
12) Let f (n) and g(n) be asymptotically nonnegative functions. Then
Max {(f (n), g (n)) = (f (n) + g (n))
For any a,b and c, where b>0 and c>1
(lg n)a nb c n
i.e., logarithm polynomial exponent
i)If a and b are real constants such that c>1

13)

nb

limn c n = 0

nb = O(c n )
Put lg n for n ,b for a and 2b for c in (i)
(lg n)a

ii) limn (2b )lg n =limn

(lg n)a = O(nb )

14) Implications of Dominance

na dominates nb if a > b since

(lg n)a
nb

=0

na

limn n b = nab 0

15) The dominance ranking of the basic functions:

n! 2n n3n2 n log n n log n 1
16) Advance Dominance ranking
2n

(c>=2, small positive real number)

n! c n n2n1+ n log n log n! n n log2 n log nlog nlog log n 1

n

Practice Problems
1. Function g is an upper bound on function f iff for all x,
(a) g(x) f (x);
(b) g(x) f (x);
(c) g = O( f ); (d) f = (g);
2. Function g is a lower bound on function f iff for all x,
(a) g(x) f (x);
(b) g(x) f (x);
(c) f = O(g);
3. Big-Omega notation expresses
(a) tight bounds; (b) upper bounds;

(d) g = (f);

(c) lower bounds;

(d) worst cases;

4. Big-O notation expresses

(a) tight bounds;

(b) upper bounds;

(c) lower bounds;

(d) best cases;

(c) lower bounds;

(d) Average cases;

5. Theta notation expresses

(a) tight bounds; (b) upper bounds;

6. Which of the following properties is/are satisfied by O-notation

I Reflexive
II Symmetric
(a) I and II only
(b) I and III only
(c) II and III only
(d) I,II,III

III Transitive

7. Which of the following properties is/are satisfied by -notation

I Reflexive
II Symmetric
(a) I and II only
(b) I and III only
(c) II and III only
(d) I, II, III

III Transitive

8. Which of the following properties is/are satisfied by -notation

I Reflexive
II Symmetric
(a) I and II only
(b) I and III only
(c) II and III only
(d) I, II, III

9. n1/ lg n is
(a) (1)

10. 4lg n is
(a) (1)

11. n2 23lg n is
(a) (n2)
12.

III Transitive

(b) (lg n)

(c) ( n)

(d) (lg n)

(b) (lg n)

(c) (n lg n)

(d) (n2)

(b) (n4)

(c) (n5)

(d) (n6)

Which of the following correctly describes the asymptotic order of growth of following
functions:
f1=lg10 n
f2=n5
f3=2n
(a) f1< f2< f3
(b) f2< f1< f3 (c) f3< f2< f1 (d) f1< f3< f2

13. Which of the following correctly describes the asymptotic order of growth of following
functions
f1=2lg n

lg n

f2=2

f3=n2

(a) f1< f2< f3

(b) f2< f1< f3 (c) f3< f2< f1 (d) f1< f3< f2
14. Which of the following correctly describes the asymptotic order of growth of following
functions
f1=4lg n
(a) f1< f2< f3

f2=n1/ lg n

lg n

f3=2

(b) f2< f1< f3 (c) f2< f3< f1

(d) f1< f3< f2

Problems:
For each of the following pairs of functions, either f(n) is O(g(n)), f(n) is (g(n)), or f(n) =
(g(n)). Determine which relationship is correct.
1) f(n) = log n2; g(n) = log n + 5
2) f(n) = n; g(n) = log n2
3) f(n) = log log n; g(n) = log n
4) f(n) = n; g(n) = log2 n
5) f(n) = n log n + n; g(n) = log n
6) f(n) = 10; g(n) = log 10
7) f(n) = 2n; g(n) = 10n2
8) f(n) = 2n; g(n) = 3n
Solutions:
1) f(n) = (g(n))
2) f(n) = (g(n))
3) f(n) = O(g(n))
4) f(n) = (g(n))
5) f(n) = (g(n))
6) f(n) = (g(n))
7) f(n) = (g(n))
8) f(n) = O(g(n))