Introducing Myself

LOW, Bryan Kian Hsiang PhD: Carnegie Mellon Univ., 2009 Undergraduate: SoC, NUS, 2001 Website: http://www.comp.nus.edu.sg/~lowkh Email: lowkh@comp.nus.edu.sg Research interests: AI, multi-agent systems, machine learning, robotics Lecture topics: counting/combinatorics, graphs, trees

Basics of Counting
Read Rosen, 5.1

What is Counting?
Combinatorics
Study of arrangement of objects

Enumeration
Counting of objects with certain properties Example: Counting the elements in a list If m and n are integers, and m n, then there are n m + 1 integers from m to n
List Count m (m+0) 0+1=1 m+1 1+1=2 m+2 2+1=3 n (m+(n-m)) (n-m)+1

Why Bother to Count?

What is the time complexity of sorting algorithm? Are there enough IP addresses or phone numbers to meet the demand for next 10 years? What are the ways to place 10 sensors in an environment discretized into a 10 10 grid? Will at least 3 out of 15 workstations be connecting to one of the 7 servers?

Major Finite Counting Methods

Basic Counting Rules
Multiplication or product rule Addition or sum rule

Permutations and Combinations

Counting problems with ordered and unordered arrangements of objects of a set

The Pigeonhole Principle

Counting multiple or repeated objects with specied properties in a set

The Inclusion-Exclusion Principle

Counts the number of elements in a union of sets

Recurrence Relations
Determine counting formula from equation relating the terms of a sequence

Generating Functions
Use formal power series to solve some of the above counting problems

Topics
Product/Multiplication Rule Sum/Addition Rule Combining Rules Inclusion-Exclusion Principle Possibility Trees

Product Rule

Product/Multiplication Rule
Suppose that an operation can be broken down into a sequence of 2 tasks: task A and task B. If there are nA ways to do rst task A and nB ways to do second task B for each way of doing task A, the number of ways to do the operation is nAnB . Can you generalize the Product Rule for multiple tasks?
8

Using Product Rule

To apply the Product Rule, Think of the objects you are trying to count as the output of a multi-task operation The possible ways to perform a task may depend on how previous tasks were performed But, number of ways to perform each task must be constant regardless of actions taken in prior tasks

Robot motion example

Example
The chairs of an auditorium are to be labeled with a letter and a positive integer not exceeding 100 What is the largest number of chairs that can be labeled differently? Solution:
Task A: Assigning one of 26 possible letters Task B: Assigning one of 100 possible integers Product Rule: 26 100 = 2600 different ways to label chairs Hence, the largest no. of chairs that can be labeled differently is 2600

10

Generalized Product Rule

Suppose that an operation can be broken down into a sequence of m tasks T1, T2, , Tm. If task Ti can be done in ni ways regardless of how tasks T1, T2, , Ti-1 were done for i = 1, , m, the number of ways to do the operation is n1n2 nm . Proof by mathematical induction. Omitted. Use the Product Rule for two tasks.
11

Example: Counting Functions

How many functions are there from a set with m elements to one with n elements? Solution:
For each of the m elements in the domain, a function corresponds to the choice of one of the n elements in the codomain. Suppose that the elements in the domain are a1,a2, , am. There are n ways to choose value of function at a1. Then there are also n ways to choose the value of function at a2 and so on. By the Product Rule, there are n n n = nm
functions from a set with m elements to one with n elements.

12

Example: Counting 1-to-1 Functions

How many one-to-one functions are there from a set with m elements to one with n elements? Solution:
When m > n, there are no one-to-one functions from a set with m elements to a set with n elements. Let m n. Suppose the elements in the domain are a1,a2, , am. There are n ways to choose value of function at a1. Since function is one-to-one, value of function at a2 can be picked in (n 1) ways, and at ak can be picked at (n k + 1) ways. By the Product Rule, there are n (n 1)(n 2) (n m + 1) one-to-one functions.

13

Representing Sets with Bit Strings

How to represent sets in a computer?
Storing set elements in an unordered fashion Storing set elements in an arbitrary order (why?)

Representing sets with ordered bit strings

Assume a nite universal set U Specify an arbitrary ordering of the set elements a1, , an Represent a subset A of U with the bit string of length n The i-th bit in the string is 1 if ai belongs to A The i-th bit in the string is 0 if ai does not belong to A
14

Example
Let U = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}. The ordering of the elements of U has the elements in increasing order, i.e., ai = i. What bit strings represent
the subset of all odd integers in U? the subset of all even integers in U? The subset of integers not exceeding 5 in U?

15

Solution
Length of the bit strings = 10 (why?) For the subset of odd integers, O = {1, 3, 5, 7, 9}
The bit string has one bit in the 1st, 3rd, 5th, 7th, and 9th positions, and a zero elsewhere, i.e., 1010101010

For the subset of even integers, E = {2, 4, 6, 8, 10}

The bit string has one bit in the 2nd, 4th, 6th, 8th, and 10th positions, and a zero elsewhere, i.e., 0101010101

For the subset of integers not exceeding 5, A = {1, 2, 3, 4, 5}

The bit string has one bit in the 1st, 2nd, 3rd, 4th, and 5th positions, and a zero elsewhere, i.e., 1111100000
16

Example
Given that the bit string of the subset O = {1, 3, 5, 7, 9} for the universal set U = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10} is 1010101010 , What is the bit string for the complement of this set? [Solution]: To obtain the complement, replace 0s with 1s and vice versa: 0101010101
which corresponds to the set E = {2, 4, 6, 8, 10} Question: How to perform intersection and union on the subsets?

17

Example: Counting Subsets of a Finite Set

Use the Product Rule to show that the number of different subsets of a nite set S is 2|S|. Solution:
Let S be a nite set. List the elements of S in arbitrary order. Recall that there is a 1-to-1 correspondence between subsets of S and bit strings of length |S| (Rosen, 2.2):
A subset of S is associated with the bit string with a 1 in the ith position if the ith element in the list is in a subset, and a 0 in this position otherwise.

By the Product Rule, there are 2|S| bit strings of length |S|. Hence, |P(S)| = 2|S|.

18

Product Rule (Set Notation)

If A1, A2, , Am are nite sets, the number of elements in the Cartesian product of these sets is the product of the number of elements in each set. Choosing an element in the Cartesian product A1 A2 Am is done by choosing an element in A1, an element in A2, , and an element in Am. By the Product Rule, it follows that | A1 A2 Am | = |A1| |A2| |Am| .
19

Sum Rule

20

Sum/Addition Rule
Suppose that an operation can be broken down into 2 tasks A and B that cannot be done at the same time. If there are nA ways to do task A and nB ways to do task B, the number of ways to do the operation is nA + nB . Can you generalize the Sum Rule for multiple tasks?
21

Example
Suppose that either a member of the computer science faculty or a student who is a computer science major is chosen as a representative to a university committee. How many different choices are there for this representative if there are 37 members of the faculty and 83 students? Solution:
The rst task, choosing a member of the faculty, can be done in 37 ways. The second task, choosing a student, can be done in 83 ways. From the Sum Rule, if follows that there are 37 + 83 = 120 possible ways to pick the representative.

22

Generalized Sum Rule

Suppose that an operation can be broken down into a number of m tasks T1, T2, , Tm. If task Ti can be done in ni ways for i = 1, , m, and no two of these tasks can be done at the same time, the number of ways to do the operation is n1 + n2 + + nm . Proof by mathematical induction. Omitted. Use the Sum Rule for two tasks.
23

Example
A student can choose a computer project from one of three lists. The three lists contain 23, 15, and 19 possible projects, respectively. How many possible projects are there to choose from? Solution: By the Sum Rule, the student can choose from 23 + 15 + 19 = 57 projects.
24

Sum Rule (Set Notation)

If A1, A2, , Am are disjoint sets, the number of elements in the union of these sets is the sum of the number of elements in each set. Let Ti be the task of choosing an element from Ai for i = 1, 2, , m. There are |Ai| ways to do Ti. By the Sum Rule, since no two tasks can be done at the same time, the number of ways to choose an element from one of the sets is | A1 A2 Am | = |A1| + |A2| + + |Am| .
25

Combining Rules

26

Combining Counting Methods

Many counting problems cannot be solved using just the Product Rule or the Sum Rule, but can be addressed using a combination of these rules. Visualization tools such as possibility trees are sometimes used to aid in the counting.

27

The Difference Rule

A consequence of the Sum Rule If A is a nite set and B is a subset of A, then |A B| = |A| - |B| . This is because
If B is a subset of A, then the two sets B and A B have no elements in common, and B (A B) = A. Hence, by the Sum Rule, |B| + |A B| = |A|. Subtracting |B| from both sides, |A B| = |A| - |B|.

28

Example: Counting Passwords

Each user on a computer system has a password, which is six to eight characters long, where each character is an uppercase letter or digit. Each password must contain at least one digit. How many possible passwords are there?

29

Example: Counting Passwords

Solution:
Let P be the total no. of possible passwords Let P6, P7, P8 be the no. of possible passwords of length 6, 7, 8 respectively By the Sum Rule, P = P6 + P7 + P8 By the Product Rule, the no. of strings of six characters is 366, and the no. of strings with no digits is 266. By the difference rule:
P6 = 366 266 = 1,867,866,560 P7 = 367 267 = 70,332,353,920 P8 = 368 268 = 2,612,282,842,880

Hence, P = 2,684,483,063,360
30

Inclusion-Exclusion Principle

31

Inclusion-Exclusion Principle
Does not assume mutually disjoint sets. If A and B and C are nite sets, then |A B| = |A| + |B| - |A B|
and |A B C | = |A| + |B| + |C| |A B| - |B C| - |A C| + |A B C | . Proof by mathematical induction. Omitted.
1
2
1

3
2

32

Example: Elements in Intersection

There are 50 students in a class:

30 know English 18 know Chinese 26 know Malay 9 know both English and Chinese 16 know both English and Malay 8 know both Chinese and Malay 47 know at least one of the three languages

1. How many students do not know any languages? 2. How many students know all three languages? 3. How many students know English and Chinese but not Malay? How many students know English but neither Chinese nor Malay?
33

Example: Elements in Intersection

1. How many students do not know any languages? By the difference rule, no. of students who do not know any languages is 50 47 = 3 .

34

Example: Elements in Intersection

2. How many students know all three languages? Let C, E, M denote the sets of students who know Chinese, English, and Malay, respectively. By the inclusion-exclusion principle,
|C E M | = |C| + |E| + |M| - |C E|- |E M|






|C M| + |C E M | 47 = 18 + 30 + 26 9 16 8 + |C E M | Hence, |C E M | = 6.
35

Example: Elements in Intersection

3. How many students know English and Chinese but not Malay? How many students know English but neither Chinese nor Malay?
Hint: Use a Venn Diagram to solve this problem
C =18 3 7 6 2 8 M =26 10 11 E =30

Answers: No. of students who know English and Chinese but not Malay = 3 No. of students who know English only = 11

36

Possibility Trees

37

Possibility Tree
Does not assume number of ways to perform each task to be independent of actions taken in prior tasks Systematically keep track of all possibilities in sequential events Branching Factors
Uniform Non-uniform

Subtrees
Balanced Unbalanced

Reordering Leaves
Final outcomes/events
38

Example
Suppose that a Gap t-shirt comes in 4 different sizes: S, M, L, and XL. Further suppose that each size comes in 3 colors: Red, White, and Blue, except for L, which comes only in Red and White. How many different shirts does a souvenir shop have to stock to have at least one of each available size and color of the T-shirt? Solution:
Answer is 11 = 3*3 + 1*2

XL

B R

W R

39

Example
Three ofcers a president, a treasurer, and a secretary are to be chosen from among 4 people: Ali, Ben, Chee, and Devi. Suppose that, for various reasons, Ali cannot be president and either Chee or Devi must be secretary. How many ways can the ofcers be chosen? Solution:
Can we use the Product Rule directly? 3 choices for president (except Ali) 3 choices for treasurer (except president) 2 choices for secretary (Chee or Devi) By Product Rule, there are 3*3*2 = 18 ways Is this correct?
40

Solution 1
There are only 8 ways to choose!

Task 1: president

Task 2: treasurer

Task 3: secretary C D D C D D C C

41

Solution 2 (Reordering)
By applying Product Rule, there are 2 2 2 = 8 ways to choose

Task 1: secretary

C B

Task 2: president

Task 3: treasurer A D A B A C A B

42

Example
How many bit strings of length 4 do not have consecutive 1s? Solution: The possibility tree displays all bit strings of length 4 without two consecutive 1s. Answer is 8

1st bit 2nd bit 3rd bit 4th bit

1 0 1 0 1 0

0 0 1 0

01

1 0

0 1 0

43

How to Avoid Double Counting?

Questions
Is anything counted twice? Is everything counted?

Product Rule
Does any outcome appear on more than one branch of the tree? Does every outcome/element appear as a branch of the tree?

Sum Rule
Do any two subsets share common elements? All the subsets must be disjoint. Does every outcome/element appear in some subset of the diagram? ??

44

Permutations
Read Rosen, 5.3, 5.5

45

Permutations
Denition. A permutation of a set of objects is an ordered arrangement of the objects. Remark. Recall that the elements of a set are distinct. Example. Permutations of {a, b, c}. How many permutations are there? 3 ways to select x {a, b, c} of xyz. 2 ways to select y {a, b, c} - {x} of xyz. 1 way to select z {a, b, c} - {x, y} of xyz.

46

Possibility Tree
Example. Permutations of {a, b, c}.
a c b

abc

acb

bac

bca

cab

cba

47

Permutations
Theorem 1. Let n be an integer with n 0. Then, the number of permutations of a set with n distinct elements is n!. Proof. 1. Each permutation is a sequence of n elements. 2. The 1st element of the sequence can be selected in n ways because there are n elements in the set. 3. In general, the k-th element of the sequence can be selected in n k + 1 ways for k = 1, . . . , n because there are n k + 1 elements left in the set. 4. The sequence of n elements can be selected in n (n 1) . . . 1 = n! ways, by the Product Rule. So, there are n! permutations.
48

r-Permutations
Denition. An r-permutation of a set of n elements is an ordered selection of r elements taken from the set of n elements. The number of r-permutations of a set of n elements is denoted P(n, r). Remark. Recall that the elements of a set are distinct.

49

r-Permutations
Theorem 2. Let n and r be integers with 0 r n. Then, the number of r-permutations of a set of n elements is given by P (n, r) = n (n 1) (n 2) . . . (n r + 1). Proof. 1. Each r-permutation is a sequence of r elements.

2. The 1st element of the sequence can be selected in n ways because there are n elements in the set. 3. In general, the k-th element of the sequence can be selected in n k + 1 ways for k = 1, . . . , r because there are n k + 1 elements left in the set. 4. By the Product Rule, the sequence of r elements can be selected in n (n 1) . . . (n r + 1) ways, which is the number of r-permutations.
50

r-Permutations
Corollary 1. Let n and r be integers with 0 r n. Then, P (n, r) = Proof. P (n, r) = n (n n (n = = n! (n r)! 1) . . . (n r + 1) 1) . . . (n r + 1)(n (n r)! r)! n! (n r)! .

51

Possibility Tree
Example. 2-Permutations of {a, b, c}. There are 3!/(3-2)! = 6 ways of arranging 2 elements of {a, b, c}.
a c b

ab

ac

ba

bc

ca

cb

52

r-Permutations
Example. There are 5 4 3 = 60 ways of arranging 3 of the 5 letters of BYTES. There are 4 3 = 12 ways of arranging 3 of the 5 letters of BYTES such that the rst letter must be B. To explain this, consider the number of ways to form the sequence Bab where a, b {Y, T, E, S}.

53

Property of P(n, r)
Corollary 2. For all integers n By Corollary , P n, and P n, So, P n, P n, nn n n . n n n n nn n nn n n nn , P n, P n, n .

n.

54

r-Permutations with Repetition

Theorem 3. Let n and r be integers with r, n 0. Then, the number of r-permutations of a set of n elements with repetition allowed is nr . Proof. 1. Since repetition is allowed, there are n ways to select an element of the set for each of the r positions in an rpermutation because all n objects are available for each choice. 2. By the Product Rule, there are nr r-permutations.

55

Possibility Tree
Example. 3-permutations of {0,1} with repetition allowed. There are 23=8 3-permutations of the set {0,1}.
0 1

000

001 010

011

100

101 110

111

56