Big O Notation

Big O Notation
Anson Ho

26 January 2019

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Motivation
• Analyse program runtime in a better way

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Motivation
• Analyse program runtime in a better way

• Learn a common language for discussion in OI

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Time Complexity
• Description of runtime (or the number of operations)

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Time Complexity
• Description of runtime (or the number of operations)

• Usually not a exact number

• But a function

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Time Complexity
• Worst case

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Time Complexity
• Worst case

• Average case (less common)

• Random algorithm
• Random test data

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Space Complexity
• Description of memory usage

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Big O Notation
• An usual representation for time/space complexity

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Big O Notation
• An usual representation for time/space complexity

• An upper bound of a function

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Big O Notation
• An usual representation for time/space complexity

• An upper bound of a function

• Constants and ”smaller” terms are usually ignored

• Therefore, it is equivalent to measure runtime or to count the number of
operations

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Big O Notation
• An usual representation for time/space complexity

• An upper bound of a function

• Constants and ”smaller” terms are usually ignored

• Therefore, it is equivalent to measure runtime or to count the number of
operations

• e.g.: 6n2 + 2n = O(n2 )

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
Advanced content (feel free to skip)
Big O Notation

Definition
f (n) = O(g(n)) if there exists n0 , c > 0 such that

f (n) ≤ cg(n) for all n ≥ n0

f (n) = Ω(g(n)) if there exists n0 , c > 0 such that

f (n) ≥ cg(n) for all n ≥ n0

f (n) = Θ(g(n)) if there exists n0 , c1 , c2 > 0 such that

c1 g(n) ≤ f (n) ≤ c2 g(n) for all n ≥ n0

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Convention
• Since Big O notation is about upper bound, 7(n + 1)2 can be represented
by O(nn )

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Convention
• Since Big O notation is about upper bound, 7(n + 1)2 can be represented
by O(nn )

• We usually use the best (”smallest”) known upper bound

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Convention
• Since Big O notation is about upper bound, 7(n + 1)2 can be represented
by O(nn )

• We usually use the best (”smallest”) known upper bound

• Therefore, the first line is a bad example

• O(n2 ) is a better representation for that

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Examples
• Linear search
O(n)

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Examples
• Linear search
O(n)

• Bubble sort
O(n2 )

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Examples
• Linear search
O(n)

• Bubble sort
O(n2 )

• Binary search
O(log n)

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Examples
• Linear search
O(n)

• Bubble sort
O(n2 )

• Binary search
O(log n)

• Merge sort
O(n log n)

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Properties
• Addition

O(f (n)) + O(g(n)) = O(f (n) + g(n))

= O(f (n)) if f (n) is ”greater” then g(n)

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Properties
• Addition

O(f (n)) + O(g(n)) = O(f (n) + g(n))

= O(f (n)) if f (n) is ”greater” then g(n)

• Scalar multiplication

O(cf (n)) = O(f (n)) if c is a constant

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Multivariable
• O(nmk)

• O(n2 m + nm2 )

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Constant
• Is O(n) always better than O(n log n)?

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Constant
• Is O(n) always better than O(n log n)?

• No, since the constants are hidden

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Constant
• Is O(n) always better than O(n log n)?

• No, since the constants are hidden

• Constant factor
• Nature of the algorithm
• Implementation
• Machine performance

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Constant
• Is O(n) always better than O(n log n)?

• No, since the constants are hidden

• Constant factor
• Nature of the algorithm
• Implementation
• Machine performance

• The constant is usually dropped but not always fully dropped

• std::bitset
• O( 32
n
)

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Amortized Complexity
• Sometimes, the time complexity of a specific function is analysed instead
of the whole program

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Amortized Complexity
• Sometimes, the time complexity of a specific function is analysed instead
of the whole program

• e.g.: precomputation, query, ...

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Amortized Complexity
• Sometimes, the time complexity of a specific function is analysed instead
of the whole program

• e.g.: precomputation, query, ...

• Amortized complexity: average performance of a function within a single

run (cf. average case)

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Amortized Complexity
• Sometimes, the time complexity of a specific function is analysed instead
of the whole program

• e.g.: precomputation, query, ...

• Amortized complexity: average performance of a function within a single

run (cf. average case)

• e.g.: std::vector, implementing queue by 2 stacks

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics
 Big O Notation

Exercises

香港電腦奧林匹克競賽
Hong Kong Olympiad in Informatics