Algorithms: Readings: (SG) Ch. 2 & 3 Chapter Outline

Algorithms
Readings: [SG] Ch. 2 & 3
Chapter Outline:
1. Chapter Goals
2. What are Algorithms
1. Real Life Examples (origami, recipes)
2. Definition
3. Example: A = B + C
3.
4.
5.
6.
Expressing Algorithms Pseudo-Code

Simple Algorithms
Recursive Algorithms
Time Complexity of Algorithms
(UIT2201: Algorithms) Page 1
LeongHW, SoC, NUS
Algorithms
Computing
devices are dumb
How to explain to a dumb mechanical /

computing device how to solve a problem
How
to solve a problem using
a small, basic set of primitive

instructions.
Complexity
of Solving Problems.
LeongHW, SoC, NUS
1. Goals of Algorithm Study
To develop framework for instructing

computer to perform tasks
To introduce notion of algorithm as means of

specifying how to solve a problem
To introduce and appreciate approaches for

defining and solving very complex tasks in
terms of simpler tasks;
LeongHW, SoC, NUS
Chapter Outline:
1. Chapter Goals
1. Real Life Examples (origami, recipes)
2. Definition of Algorithm
3. Example: A = B + C
3.
4.
5.
6.

Simple Algorithms
LeongHW, SoC, NUS
2. Computer Science and Algorithms

Computer
Science can be considered
as study of algorithms including

their formal properties
Their hardware and software realisations
Their applications to diverse areas
LeongHW, SoC, NUS
Algorithms: Real Life Examples

Many
Real-Life Examples
Cooking: Recipe for preparing a dish

Origami: The Art of Paper Folding
Directions: How to go to Changi Airport
Remember:
Framework: How to give instructions;

Alg: The actual step-by-step instructions;
Abstraction: Decomposing / Simplifying
LeongHW, SoC, NUS
Real Life Example: Cooking

Framework:
Cooking or Recipe language

Algorithm:
Recipe for a Dish

(step by step instructions on how to cook a dish)
Problem
Decomposition
Preparing Ingredients;
Preparing Sauce;
LeongHW, SoC, NUS
Real Life Example: Origami

Framework:
Origami or Paper-Folding language

Algorithm:
Sequence of Paper-Folding Instructions

(step by step instructions for each fold)
Problem
Decomposition
Start with a Bird Base;

Finish the Head;
Finish the Legs; Finish the Tail;
LeongHW, SoC, NUS
Real Life Examples: Issues

Problems/Difficulties:
Imprecise Instructions;
Job can often be done even if instructions are
not followed precisely
Modifications may be done by the person
following the instructions;
But,
NOT for a Computer
Needs to told PRECISELY what to do;

Instructions must be PRECISE;
Cannot be vague or ambiguous
LeongHW, SoC, NUS
Definition of Algorithm:
An
algorithm for solving a problem
a finite sequence of unambiguous, executable

steps or instructions, which, if followed would
ultimately terminate and give the solution of
the problem.
Note the keywords:
Finite set of steps;

Unambiguous;
Executable;
Terminates;
LeongHW, SoC, NUS
Algorithm Examples?
Problem 1: What is the largest integer

INPUT: All the integers { -2, -1, 0, 1, 2, }
OUTPUT: The largest integer
Algorithm:
Arrange all the integers in a list in decreasing order;
MAX = first number in the list;
Print out MAX;
WHY is the above NOT an Algorithm?

(Hint: How many integers are there?)
Problem 2: Who is the tallest women in the world?

Algorithm: Tutorial...
LeongHW, SoC, NUS
Example: Adding two (n-digit) numbers

Input: Two positive m-digit decimal numbers
(a and b)
am, am-1, ., a1
bm, bm-1, ., b1
Output: The sum c = a + b
cm+1, cm, cm-1, ., c1

(Note: In the textbook, it was am-1,,a1,a0 )
LeongHW, SoC, NUS
How to derive the algorithm

Adding
is something we all know
done it a thousand times, know it by heart

How
do we give the algorithm?
A step-by-step instruction
to a dumb machine
Try
an example:
3492
8157
Imagine you looking at yourself solving it
LeongHW, SoC, NUS
Algorithm: Finding sum of A & B

Step 1: Set the value of carry to 0
Step 2: Set the value of i to 1.
Step 3: Repeat steps 4 through 6 until the value of i is > m.
Step 4: Add ai and bi to the current value of carry, to get xi.
Step 5: If xi < 10 then
let ci=xi and reset carry to 0.
Else (* i.e. xi 10 *)
let ci=xi - 10 and reset carry to 1.

Step 6: Increase the value of i by 1.
Step 7: Set cm+1 to the value of carry.

Step 8: Print the final answer cm+1, cm, ., c1
Step 9: Stop.
LeongHW, SoC, NUS
Chapter Outline:
1. Chapter Goals
3. Expressing Algorithms Pseudo-Code
1.
2.
3.
4.
5.
Communicating Alg to computer

Pseudo-Code
Primitive Operations and examples
Variables and Arrays
Algorithm C=A+B in pseudo-code
4. Simple Algorithms
5. Recursive Algorithms
6. Time Complexity of Algorithms
LeongHW, SoC, NUS
3. Expressing Algorithms to Computer

To
communicate algorithm to computer
Need way to represent the algorithm

Cannot use English
Can
use computer language
machine language and

programming languages (Java, Pascal, C)
But, these are too tedious (&technical)
Use
Pseudo-Code instead
LeongHW, SoC, NUS
Pseudo-Code to express Algorithms

Pseudo-Code
Mixture of computer language and English

Somewhere in between
precise enough to describe what is meant without
being too tediuos
Examples:
Let c be 0;
Sort the list of numbers in increasing order;
Need
to know both
syntax representation
semantics meaning
LeongHW, SoC, NUS
Primitive Operations
To
describe an algorithm, we need some

well-defined programming primitives
Assignment primitive:
assignment statement, input/output statements
Conditional primitive:
if statement
case statement
Looping (iterative) primitive:

for loop,
while loop,
Statements
are executed one-by-one

LeongHW, SoC, NUS
Conditional Primitives
if
statement
to take different actions based on condition

Syntax
if (condition)
then (Step A)
else (Step B)
endif
true
Step A
condition?
false
Step B
if (condition)
then (Step A)
endif
Semantics
LeongHW, SoC, NUS
Examples -- (if statement)

Let mark be the total-mark obtained
if (mark < 40)
then (print Student fail)
else (print Student pass)
endif
read in mark (*from the terminal*)

if (mark < 40) then (Grade F)
else if (mark < 50) then (Grade
endif
print Student grade is, Grade
D)
C)
B)
A);
LeongHW, SoC, NUS
Looping Primitive while-loop
the while-loop
loop a variable
number of times
Syntax
while (condition) do
(some sequence
of statements)
condition?
false
true
Some sequence
of statements;
endwhile
Semantics
LeongHW, SoC, NUS
Looping Primitive for-loop
First, the for-loop

loop a fixed or (predetermined) number of
times
Syntax
for j a to b do
(some sequence
of statements)
j a;
(j <= b)?
false
true
Some sequence
of statements;
endfor
Semantics
j j+1;
LeongHW, SoC, NUS
Exercising the alg: for and while

for j 1 to 4 do
print 2*j;
endfor
print --- Done ---
Output:
2
4
6
8
--- Done ---
j 1;
while (j <= 4) do
print 2*j;
j j + 1;
endwhile
print --- Done ---
Output:
2
4
6
8
--- Done --(UIT2201: Algorithms) Page 23
LeongHW, SoC, NUS
Variables and Arrays

In
the computer, each variable is assigned

a storage box
can store one number at any time
eg: sum, j, carry
Arrays:
Often deal with many numbers

Such as A1, A2, A3, , A100
Store as an array A[1], A[2], , A[100]
we treat each of them as a variable,
each is assigned a storage box
LeongHW, SoC, NUS
Algorithm: A = B + C (in pseudo-code)

We can re-write the C=A+B algorithm as follows:
Alg. to Compute C = A + B:
(*sum two big numbers*)
carry 0;
for i 1 to m do
x[i] a[i] + b[i] + carry ;
if (x[i] < 10)
then ( c[i] x[i]; carry 0; )
else ( c[i] x[i] 10; carry 1; )
endfor;
c[m+1] carry;
Print c[m+1], c[m], ., c[1]
LeongHW, SoC, NUS
Chapter Outline:
1.
2.
3.
4.
Chapter Goals
What are Algorithms
Simple Algorithms
1. Simple iterative algorithms
Computing Sum, Find Max/Min
2. Modular Program Design
3. Divisibility and Prime Numbers
5. Recursive Algorithms
LeongHW, SoC, NUS
Simple iterative algorithm: Sum
Given: List of numbers: A1, A2, A3, ., An
Output: To compute the sum of the numbers
Note: Store numbers in array A[1], A[2], , A[n]

Sum(A, n);
begin
Sum_sf 0;
k 1;
while (k <= n) do
Sum_sf Sum_sf + A[k];
k k + 1;
endwhile
Sum Sum_sf;
Print Sum is, Sum
end;
LeongHW, SoC, NUS
Exercising Algorithm Sum:

Input:
A[1] A[2] A[3] A[4] A[5] A[6]

2
5
10
3
12
24
Processing:
Output:
k
?
1
2
3
4
5
6
6
Sum-sf
0
2
7
17
20
32
56
56
n=6
Sum
?
?
?
?
?
?
?
56
Sum is 56
LeongHW, SoC, NUS
Algorithm for Sum (with for-loop)
We can also use a while-loop instead of a for loop.

Sum(A, n);
(* Find the sum of A1, A2,, An. *)
begin
Sum_sf 0;
for k 1 to n do
Sum_sf Sum_sf + A[k];
endfor
Sum Sum_sf;
Print Sum is, Sum
end;
HW: (a) Note the differences

(b) Modify it to compute the average?
LeongHW, SoC, NUS
Remarks about the iterative algorithm
Note the three stages:

1. Initialization
Set some values at the beginning
2. Iteration
This is the KEY STEP
Where most of work is done
3. Post-Processing or Cleanup
Can use this setup for other problems

Calculating average, sum-of-squares
Finding max, min; Searching for a number,
LeongHW, SoC, NUS
Modular Program Design
Software are complex

HUGE (millions of lines of code) eg: Linux, Outlook
COMPLEX; eg: Flight simulator
Idea: Divide-and-Conquer Method

Complex tasks can be divided and each part solved
separately and combined later.
Modular Program Design

Divide big programs into smaller modules
The smaller parts are
called modules, subroutines, or procedures
Design, implement, and test separately
Modularity, Abstraction, Division of Labour

Simplifies process of writing alg/programs
LeongHW, SoC, NUS
Simple Algorithms: Prime Numbers
Algorithm to determine if n is prime

Given: A positive integer n
Question: Is n a prime number?
What do we know?
A number n is prime iff
n has no positive factors except 1 and n
Idea: Express it algorithmically

n is not divisible by any positive number k
such that 1 < k < n. or k=2,3,4,,(n-1)
What we already have:

A module Divisible(m,n) to check divisibility
We can use it as a primitive
LeongHW, SoC, NUS
Pseudo-Code for Prime:

Prime(n)
(* To determine if n is prime *)
begin
for k 2 to (n-1) do
if Divisible(k,n)
then (Print False & exit)
else (* Do nothing *)
endfor
Print True & Stop
end;
Note: Prime uses the module Divisible(k,n)
Exercise it with:
Prime (5); Prime (4);
Prime (55); Prime (41);
LeongHW, SoC, NUS
A Simple Module: Divisibility

A
module (procedure) for divisibility
Given: Two positive integers m and n

Question: Is n divisible by m?
or Algorithm to compute Divisible(m,n)
Divisible(m, n)
Algorithm
{False, if n is not divisible by m

True,
if n is divisible by m
Idea:
A positive integer n is divisible by m iff

for some positive integer k n, m*k = n.
LeongHW, SoC, NUS
Module for Divisibility:
Algorithm (in pseudo-code)
Exercise it with:
Divisible (3, 9);
Divisible (3, 5);
Divisible (3,101);
Divisible(m,n)
(* To compute if n is divisible by m *)
begin
D false; (* assume false, first *)
for k 1 to n do
if (m*k = n) then (Dtrue; exit-loop)
else (* Do nothing *)
endfor
Divisible D; (* true or false *)
end;
LeongHW, SoC, NUS
Chapter Outline:
1.
2.
3.
4.
5.
6.
Chapter Goals
What are Algorithms
Simple Algorithms
1. Sequential search algorithm
2. Binary search algorithm
3. Analysis of Time Complexity
7. Summary
LeongHW, SoC, NUS
Search: sequential search algorithm

List of numbers: A1, A2, A3, ., An
and a query number x;
Given:
Question: Search for x in the list;

Sequential-Search(A, n, x);
(* Search for x in A1, A2,, An. *)
begin
for k 1 to n do
if (x = A[k])
then (Print Yes; Stop)
else (* do nothing *)
endfor
Print No; Stop
end;
LeongHW, SoC, NUS
Remarks on Sequential Search Alg
Analogy: House-to-house search
How fast is it to search for x?

How many comparisons do we need?
Example: 13, 38, 19, 74, 76, 14, 12, 38, 22, 55
When x=14, need 6 comparisons
When x=13, need only 1 comparison BEST CASE
When x=55, need 10 comparisons WORST CASE
When x=5, need 10 comparisons WORST CASE
In general, given n numbers, A1,,An

Best Case: 1 comparison
Worst Case: n comparisons
Average Case: (n+1)/2 comparisons
LeongHW, SoC, NUS
Binary Search
If
the List is sorted, that is

A1 A2 A3 . An
Then,
we can do better,
actually a lot better.
LeongHW, SoC, NUS
Binary Search
1, 4, 9, 11, 14, 43, 78
LeongHW, SoC, NUS
Binary Search (How fast is it?)

Imagine
n=100
After 1 step, size is 50

After 2 steps, size is 25
DONE!!
What
if n=1000? 1,000,000?
LeongHW, SoC, NUS
Binary Search Algorithm

Sorted List A1 A2 A3 . An
A number x.
Question: Is x in the list?
Binary-Search(A,n,x);
1. First 1
Last n
2. While ( First Last ) do
mid (first+last) / 2
If ( x = A[mid] )
then (output Yes and Stop)
else If ( x < A[mid] )
then Last mid-1
else If ( x > A[mid] ) then First mid+1
EndWhile
3. Output False and Stop
4. End
Input:
LeongHW, SoC, NUS
Time Complexity Analysis

Sequential
Search (Alg):
Worst Case:
n comparisons
Avg Case: n/2 comparisons
Binary
Search (Alg):
Worst Case: log2 n comparisons

Avg Case: about log2 n comparisons
How
to get the Average Case?
using mathematical analysis

LeongHW, SoC, NUS
Complexity of Algorithm
Logarithmic Time Algorithm

Binary Search
A Linear Time Algorithm

Algorithm Sum(A,n) -- O(n) time
Algorithm Sequential-Search(A,n,x) O(n) time
A Quadratic Time Algorithm

Simple Median-Find (T2-Q3)
An Exponential Time Algorithm

All-Subsets(A,n)
LeongHW, SoC, NUS
Chapter Outline:
1.
2.
3.
4.
5.
Chapter Goals
What are Algorithms
Simple Algorithms
1. Recursion the idea
2. Fibonacci sequence
3. Tower of Hanoi
LeongHW, SoC, NUS
5. Recursion
A problem solving method of

decomposing bigger problems into
smaller sub-problems that are identical to itself.
General Idea:
Solve simplest (smallest) cases DIRECTLY
usually these are very easy to solve
Solve bigger problems using smaller sub-problems

that are identical to itself (but smaller and simpler)
Abstraction:
To solve a given problem, we first assume that we
ALREADY know how to solve it for smaller instances!!
Dictionary definition:
recursion
see recursion
LeongHW, SoC, NUS
Example: Fibonacci Numbers
Definition of Fibonacci numbers

1. F1 = 1,
2. F2 = 1,
3. for n>2, Fn = Fn-1 + Fn-2
Problem: Compute Fn for any n.
The above is a recursive definition.

Fn is computed in-terms of itself
actually, smaller copies of itself Fn-1 and Fn-2
Actually, Not difficult:

F3 = 1 + 1 = 2
F4 = 2 + 1 = 3
F5 = 3 + 2 = 5
F6 = 5 + 3 = 8
F7 = 8 + 5 = 13
F8 = 13 + 8 = 21
F9 = 21 + 13 = 34
F10 = 34 + 21 = 55
F11 = 55 + 34 = 89
1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144,

LeongHW, SoC, NUS
Fibonacci Numbers: Recursive alg

Fibonacci(n) (* Recursive, SLOW *)
begin
if (n=1) or (n=2)
then Fibonacci(n) 1 (*simple case*)
else Fibonacci(n) Fibonacci(n-1) +
Fibonacci(n-2)
endif
end;
The
It
above is a recursive algorithm
is simple to understand and elegant!
But,
very SLOW
LeongHW, SoC, NUS
Recursive Fibonacci Alg -- Remarks

How
slow is it?
Eg: To compute F(6)

F(6)
F(5)
F(4)
F(4)
F(3)
F(2)
F(3)
F(2)
F(2)
F(3)
F(1)
F(2)
F(2)
F(1)
F(1)
HW: Can we compute it faster?

LeongHW, SoC, NUS
Example: Tower of Hanoi

A
Given: Three Pegs A, B and C

Peg A initially has n disks, different size, stacked up,
larger disks are below smaller disks
Problem: to move the n disks to Peg C, subject to

1. Can move only one disk at a time
2. Smaller disk should be above larger disk
3. Can use other peg as intermediate
LeongHW, SoC, NUS
Tower of Hanoi
How to Solve: Strategy

Generalize first: Consider n disks for all n 1
Our example is only the case when n=4
Look at small instances

How about n=1
Of course, just Move disk 1 from A to C
How about n=2?

1. Move disk 1 from A to B
2. Move disk 2 from A to C
3. Move disk 1 from B to C
LeongHW, SoC, NUS
Tower of Hanoi (Solution!)
General Method:
First, move first (n-1) disks from A to B
Now, can move largest disk from A to C
Then, move first (n-1) disks from B to C
Try this method for n=3

2. Move disk 2 from A to B
3. Move disk 1 from C to B
5. Move disk 1 from B to A

6. Move disk 1 from B to C
LeongHW, SoC, NUS
Algorithm for Towel of Hanoi (recursive)
Recursive Algorithm
when (n=1), we have simple case
Else (decompose problem and make recursive-calls)
Hanoi(n, A, B, C);
(* Move n disks from A to C via B *)
begin
if (n=1) then Move top disk from A to C
else (* when n>1 *)
Hanoi (n-1, A, C, B);
Move top disk from A to C
Hanoi (n-1, B, C, A);
endif
end;
LeongHW, SoC, NUS
Characteristics of Algorithm
Correctness
Complexity --- time, space (memory), etc
Ease of understanding
Ease of coding
Ease of maintainence
LeongHW, SoC, NUS
If
you are new to algorithms
read the textbook

try out the algorithms
do the exercises
The End
LeongHW, SoC, NUS

Algorithms: Readings: (SG) Ch. 2 & 3 Chapter Outline

Caricato da

Informazioni sul documento

Descrizione originale:

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Algorithms: Readings: (SG) Ch. 2 & 3 Chapter Outline

Caricato da

Copyright:

Formati disponibili

Algorithms

Readings: [SG] Ch. 2 & 3

Expressing Algorithms Pseudo-Code

LeongHW, SoC, NUS

devices are dumb

How to explain to a dumb mechanical /

to solve a problem using

a small, basic set of primitive

LeongHW, SoC, NUS

1. Goals of Algorithm Study

To develop framework for instructing

To introduce notion of algorithm as means of

To introduce and appreciate approaches for

(UIT2201: Algorithms) Page 3

LeongHW, SoC, NUS

Expressing Algorithms Pseudo-Code

(UIT2201: Algorithms) Page 4

LeongHW, SoC, NUS

2. Computer Science and Algorithms

Science can be considered

as study of algorithms including

(UIT2201: Algorithms) Page 5

LeongHW, SoC, NUS

Algorithms: Real Life Examples

Cooking: Recipe for preparing a dish

Framework: How to give instructions;

(UIT2201: Algorithms) Page 6

LeongHW, SoC, NUS

Real Life Example: Cooking

Cooking or Recipe language

Recipe for a Dish

(UIT2201: Algorithms) Page 7

LeongHW, SoC, NUS

Real Life Example: Origami

Origami or Paper-Folding language

Sequence of Paper-Folding Instructions

Start with a Bird Base;

LeongHW, SoC, NUS

Real Life Examples: Issues

NOT for a Computer

Needs to told PRECISELY what to do;

(UIT2201: Algorithms) Page 9

LeongHW, SoC, NUS

algorithm for solving a problem

a finite sequence of unambiguous, executable

Note the keywords:

Finite set of steps;

LeongHW, SoC, NUS

Problem 1: What is the largest integer

WHY is the above NOT an Algorithm?

Problem 2: Who is the tallest women in the world?

LeongHW, SoC, NUS

Example: Adding two (n-digit) numbers

cm+1, cm, cm-1, ., c1

LeongHW, SoC, NUS

How to derive the algorithm

is something we all know

done it a thousand times, know it by heart

do we give the algorithm?

LeongHW, SoC, NUS

Algorithm: Finding sum of A & B

let ci=xi - 10 and reset carry to 1.

read in mark (from the terminal)