Introduction to Automata Theory

What is Automata Theory?

• Study of abstract computing devices, or
“machines”
• Automaton = an abstract computing device
▫ Note: A “device” need not even be a physical
hardware!
• A fundamental question in computer science:
▫ Find out what different models of machines can do
and cannot do
▫ The theory of computation
• Computability vs. Complexity
• 1- Automata and formal language.
• Which answers - What are computers (Or what
are models of computers)
• 2- Compatibility.
• Which answers - What can be computed by
computers?
• 3- Complexity.
• Which answers - What can be efficiently
computed?

• In automata we will simulates parts of

computers.
• Helps in design and construction of different
software's and what we can expect from our
software's.
• Automata play a major role in theory of
computation, compiler design, artificial
intelligence.
A simple computer

BATTERY

input: switch
output: light bulb
actions: flip switch
states: on, off
A simple “computer”

BATTERY start off on

input: switch
output: light bulb
actions: f for “flip switch”
states: on, off
bulb is on if and only if
there was an odd number
of flips
Another “computer”
1
1 start off off
1

2 2 2 2
BATTERY
1
2
off on
1

inputs: switches 1 and 2

actions: 1 for “flip switch 1” bulb is on if and only if
actions: 2 for “flip switch 2” both switches were flipped
an odd number of times
states: on, off
Objective
• Study of set of abstract
machines that serve as models
for computation
▫ Finite automata
▫ Pushdown automata
▫ Turing machines
 9

(A pioneer of automata theory)

Alan Turing (1912-1954)

• Father of Modern Computer
Science
• English mathematician
• Studied abstract machines called
Turing machines even before
computers existed
• Hypothetical device representing
a computation machine
 10

Theory of Computation: A Historical

Perspective
1930s • Alan Turing studies Turing machines
• Decidability
• Halting problem

1940-1950s • “Finite automata” machines studied

• Noam Chomsky proposes the
“Chomsky Hierarchy” for formal
languages

1969 Cook introduces “intractable” problems

or “NP-Hard” problems

1970- Modern computer science: compilers,

computational & complexity theory evolve
Machines
• What Different Models Of Machines Can Do And
Cannot Do
• For this purposes , we will
▫ Build mathematical models called “machines” to
represent computers
▫ Study their limitations by analyzing the types of
inputs on which they can successfully operate
Language
• Collection of successful inputs will be referred to
as the “language” of that machine
• Concept analogous to humans, who can
understand instructions in one language but not
in another
• What about programming languages and their
compilers?
Machine/Language Pairs
• Problems will be investigated by
▫ Finding a language for a machine
▫ Finding a machine corresponding to a language
Languages in the Abstract
• Entities in language
▫ Letters
▫ Words
▫ Sentences
Languages in the Abstract
• Not all collections of letters form valid words
• Not all collections of words form valid sentences
• And so on, in the case of paragraphs, stories …
• To a large degree, we agree on which sequences
are valid

How do we decide what

sequences are valid?
Similarly, in Computer Languages
• Some character strings are recognizable words
(IF, ELSE, WHILE)
• Some sequences of words are recognized as
commands
• A specific set of commands is a program (that
can be compiled)
Langauges in the Abstract
• We need a general theory
• For this, we need a definition of “language
structure”
• Such that the decision on whether a given
sequence of units is valid is not left to guess
work
• Instead based on explicitly stated rules
Important that the program
compiles, whether or not it does
what the programmer intended
Languages
• Similar to how sets work
• Languages should be able to tell what belongs
and what does not
Languages
• Hard to state all the rules for “spoken English”
or “spoken Urdu”
• Many seemingly incoherent strings of words are
actually understandable utterances because of
▫ Slang
▫ Idiom
▫ Dialect
▫ Our ability to interpret poetic metaphor
▫ Our ability to correct unintentional grammar errors
that we hear
General Theory of Abstract Languages
• For this, we need precise rules because
computers are not as robust at their jobs as
human listeners are
Theory of "Formal" Languages
• All the rules for the language are explicitly stated
in terms of what strings of symbols can occur
• No liberties are tolerated
• No references to any deeper understanding are
required
Theory of "Formal" Languages
• Considered solely as symbols on paper
• Not as expressions of ideas in human minds
• The “form” of the strings will be important, and
not the “meaning”
 24

The Central Concepts of

Automata Theory
Terminology
• Begin with one set of finite set of fundamental
units out of which structures are built, known as
the alphabet
• A certain specified set of strings of characters
from the alphabet will be known as the
language
• Permissible strings in the language will be
known as words
Alphabet
An alphabet is a finite, non-empty set of symbols
• We use the symbol ∑ (sigma) to denote an
alphabet
• Examples:
▫ Binary: ∑ = {0,1}
▫ All lower case letters: ∑ = {a,b,c,..z}
▫ Alphanumeric: ∑ = {a-z, A-Z, 0-9}
▫ DNA molecule letters: ∑ = {a,c,g,t}
▫ …
Constraints
• Symbols in the alphabet need not be Latin letters
• The only requirement is that it should contain a
finite number of symbols
Null String
• A string with no letters is conceptually allowed
• Referred to as a null string or an empty
string
• Denoted by the symbol λ
• λ however is never part of the alphabet for any
language
Terminolgy
• Word that has no letters: λ

▫ Versus

• Language that has no words: Φ

Terminology
• λ is not a word in Φ since Φ has no words
• If a language, L, does not contain λ, to add it we
use the “union of sets” operation denoted by +
• L + {λ} ≠ L
• L + Φ = L (since no new words have been added)
• Note that the language Φ exists even though it
has no words
Example: English
• Usual set of letters plus apostrophe and hyphen
(minimal set)
• ∑ = {a b c d e … z ‘ -}
• ∑ is used to denote the collection of letters that
forms the alphabet of a language
Example: English
• Specify which strings of these letters are valid
words in this language,
by listing them
• Long but finite list
• Refer to the language as ENGLISH-WORDS

ENGLISH-WORDS = {all the words in a standard dictionary}

Defining Languages
• Defining a language with an infinite dictionary is
not allowed
• Such languages can instead be defined through a
description
Defining Languages
• For a formal definition of the language of the
sentences in English

▫ Our basic alphabet would be the words in the

dictionary, denoted by Г (capital gamma)

▫ Г = {the entries in a standard dictionary plus a

blank space, plus the usual punctuation marks}
Defining Languages
• Which strings from Г produce valid words in the
language ENGLISH-SENTENCES
• Definition needs to be via grammatical rules of
English, as a list of all words in this language cannot
be produced (i.e., all possible English sentences)
• Infinitely many words in this language
▫ I ate one apple
▫ I ate two apples
▫ I ate three apples
▫ …
Defining Languages
• The solution is to list all the grammatical rules of
English
• Under the grammatical rules, many strings are
valid
I ate three Tuesdays

In a formal language, this is perfectly acceptable as

it is grammatically correct (even though it does not
have a sensible meaning)
Defining Languages
• Abstract languages will therefore be defined in
one of these two ways
▫ Presented as an alphabet and the exhaustive list of
all valid words
▫ Presented as an alphabet and a set of rules
defining the acceptable words (referred to as
grammars)
Defining Languages
• To be an acceptable specification of a language, a
set of rules must enable us to determine in a
finite amount of time, whether a given string of
alphabet letters is or is not a word in the
language
Example
• Language = MY-PET
• Alphabet for this language = {a c d g o t}
• Specification
▫ If the Earth and Moon ever collide, then
 MY-PET = {cat}
▫ But, if the Earth and Moon never collide, then
 MY-PET = {dog}
One or the other of these two events will occur,
but at this point in the history of the universe it is
impossible to be certain whether the word dog is
or is not in the language MY-PET.
Example

This sentence is not an adequate specification

of the language MY-PET because it is not
useful. To be an acceptable specification of a
language, a set of rules must enable us to
decide, in a finite amount of time, whether a
given string of alphabet letters is or is not a
word in the language.
Note
• All letters in the alphabet do not need to appear
in the words selected for the language
• For example, letters in the English alphabet that
have been discarded
• These can be added back, but the language
ENGLISH-WORDS will remain unchanged
Defining Languages
• Two kinds of language defining rules
▫ Either a test for a string to check if it is a valid
word
▫ Clear procedure for constructing all the words in
the language
Example: Language 1
• Given an alphabet with only one letter, x
Σ = {x}
• Language definition: Any non-empty string of
alphabet characters
• L1 = {x xx xxx xxxx …}
Or
• L1 = {xn for n = 1 2 3 …}
• The definition does not include the null string, λ
Concatenation Operation
• Can be defined on L1
• xxx + xx = xxxxx
• x n + xm
Using Symbols
• Useful for designating words
• Example
▫ xxx = a
▫ xx = b
▫ Concatenation = ab = xxxxx
Defining Languages – Example
Language 2
• Two concatenated words do not always produce
a word that exists in the language
• L2 = {x xxx xxxxx xxxxxxx …}
= {xodd}
= {x2n+1 for n = 0 1 2 3 …}
• a = xxx, b = xxxxx
• ab = xxxxxxxx does not exist in L2
• Note that here, ab = ba, but this does not hold
true for all languages
Example: Language 3
• ∑ = {0 1 2 3 4 5 6 7 8 9}
• L3 = {any finite string of alphabet letters that
does not start with the letter zero}
• L3 = {1 2 3 4 5 6 7 8 9 10 11 12 …}
• Visually looks like the set of all positive integers
(actually only a formal collection of strings of
symbols)
• To include 0, L3 = {any finite string of alphabet
letters that, if it starts with a zero, has no more
letters after the first}
Example: Language 3
• Alternate definition of L3

• L3 = {any finite string of alphabet letters that if it

has length more than 1, does not start with a 0}

Therefore, there are often different ways of

specifying the same language
Example: Language 4
• The term “any finite string” is vague
• Does it include λ or not?
• L3 does not include λ
• But something similar to L1 but with λ can be
defined as
▫ L4 = {λ x xx xxx xxxx …}
▫ = {xn for n= 0 1 2 3 …}
 Not to be interpreted algebraically
 0 and λ are distinct from each other; 0 is a string of
length 1
Remember, even when λ is a word in the language, it is not a
letter in the alphabet.
Reverse Operation
DEFINITION
If a is a word in some language L, then reverse(a) is
the same string of letters spelled backward, called
the reverse of a, even if this backward string is not a
word in L.
• reverse(xxx) = xxx
• reverse(xxxxx) = xxxxx
• reverse(145) = 541
• In L3
▫ Reverse(140)=041
▫ Which is not a word in L3
Example: PALINDROME
• Define a new language called PALINDROME over
the alphabet.
▫ ∑={ab}
• PALINDROME = {λ and all strings such that
reverse(x)=x}
• PALINDROME
= {λ a b aa bb aaa aba bab bbb aaaa abba …}
• Does concatenation of words in PALINDROME
result in words that are included in
PALINDROME?
Example: PALINDROME
• abba + abbaabba
= abbaabbaabba

• aa + aba
= aaaba
Kleene Closure
• Given an alphabet ∑ we wish to define a
language in which any string from letters of ∑ is
a word, even the null string
• This language is referred to as the closure of the
alphabet
• Denoted by ∑*
• Also referred to as the Kleene star
Example 1
• If ∑ = {x}

• Then ∑* = L4 = {λ x xx xxx xxxx …}

Example 2
• If ∑ = {0 1}

• Then ∑* = {λ 0 1 00 01 10 11 000 001 …}

Example 3
• If ∑ = {a b c}

• ∑* = {λ a b c aa ab ac ba bb bc ca cb cc
aaa …}
Kleene Star
• An operation that makes an infinite language of
strings of letters out of an alphabet
• Note that the words are sequenced by size, in
lexicographic order (size order-all words of the
same length alphabetically)
• Normal alphabetical ordering would not enable
a listing that would inform us of the nature of
the language
• In the case of the last example {λ a aa aaa
aaaa}
Generalizing *
DEFINITION
If S is a set of words, then by S* we mean the set
of all finite strings formed by concatenating
words from S, where any word may be used as
often as we like, and where the null string is also
included.
Example 4
• S = {aa b}
• S* = {λ plus any word composed of factors of aa
and b}
• = {λ plus all strings of a’s and b’s in which the a’s
occur in even clumps}
• = {λ b aa bb aab baa bbb aaaa aabb baab
bbaa bbbb aaaab aabaa aabbb baaaa baabb
bbaab bbbaa bbbbb …}
• Is aabaaab in S*?
• No (aabaaab)
Example 5
• S = {a ab}
• S* = {λ plus any word composed of factors of a and ab}
• = {λ plus all strings of a's and b's except those that start
with b and those that contain a double b}
• = {λ a aa ab aaa aab aba aaaa aaab aaba abaa abab
aaaaa aaaab aaaba aabab abaaa abaab ababa ... }
• Is abaab part of S*?
• To prove that abaab is in S*, factor it as
▫ (ab)(a)(ab)
▫ This is the only way this string can be factored, so it is
referred to as a unique factorization
Factoring
• Factoring is not always unique
• S = {xx xxx}
• S* = {λ and all strings of more than 1 x}
• = { xn for n = 0 2 3 4 5 ... }
• = {λ xx xxx xxxx xxxxx xxxxxx ... }
• xxxxxxx
▫ (xx)(xx)(xxx)
▫ (xx)(xxx)(xx)
▫ (xxx)(xx)(xx)
• x6 = x2x2x2 or x3x3
Factoring
• The parentheses () in the previous example were
not letters of the alphabet
• There can be cases where they are
• ∑={x()}
▫ length(xxxxx)
=5
▫ length((xx)(xxx))
=9
Proof by Constructive Algorithm

The method of proving something exists by

showing how to create it
Proof by Constructive Algorithm
• For S = {xx xxx} show that S* contains all xn for
n ≠1
• Consider the possibility of whether some powers
of x cannot be produced by concatenating factors
of (xx) and (xxx)
• We know x4, x5, x6 can be produced, so consider
larger examples.
Proof by Constructive Algorithm
• What is the smallest power of x (larger than 1)
that we cannot produce?
• Start making factorized list x2, x3, x4, x5
• Lets say we get stuck at x374
• Solution
▫ We do have x372, so just add (xx)
▫ This can then be generalized for all powers of x
greater than 1
Proof by Constructive Algorithm

• Proof based on showing that something exists

• Because we can describe how to create it (by
adding xx to a previous case)
Formal Algorithm
• Formal algorithm for producing all the powers of
x from the factors xx and xxx
• Begin with xx and xxx
• To produce xn, take the sequence of
concatenations that we already know will
produce xn-2
• Concatenate xx onto that
Alphabet with No Letters
• Example
▫ If ∑ = Φ (the empty set)
▫ Then ∑* = {λ} (because λ is always included in *)
• Different from
▫ If S = {λ}
▫ Then S* = {λ}
▫ This is because λλ = λ
• The Kleene closure always produces an infinite
language, with the exception of these two
examples
Example 6
• Consider the two languages
▫ S = { a b ab }
▫ T = { a b bb }
• Both S* & T* are the languages of all strings of
a's and b's
• How will you prove this?

This is because any string of a's and b's can be

factored into syllables of either (a) or (b) both of
which are in S and T
Closure Modification
• Concatenation of non-empty strings (do not
include λ)
• Use + notation instead of *
• ∑ = {x}
• ∑+ = {x xx xxx ...}
• Identical to L1 discussed earlier
Closure Modification
• If S is a set of strings not including λ, then S+ is
the language S* without the word λ
• If T is a set of letters, then T+ means the same as
T* except that it does not include λ
• If S contains λ then S+ = S*
• + operation known as positive closure
Positive Closure
• If S = { xx xxx }
• Then S+ is the same as S* except for the word λ
• And S+ can contain λ but only if S contains the
word λ initially
Example 7
• S = {w1 w2 w3}
• S+ = {w1 w2 w3 w1w1 w1w2 w1w3 w2w1 w2w2
w2w3 w3w1 w3w2 w3w3 w1w1w1 w1w1w2 … }

• No matter what the words are

• If w1 = aa, w2 = bbb and w3 = λ
• S+ = {aa bbb λ aaaa aabbb …}
• Order is as per the w sequencing
Closure of Closure
• Apply twice
• S
• S*
• (S*)* or S**
Closure of Closure
• Unless S is empty, or it contains only λ, S* will
be infinite.
• So S** is the closure of an infinite set.
• Since every string is a combination of a finite
number of words.
• Therefore, closure operator can apply to infinite
sets as well.
Theorem

For any set S of strings, S* = S**

Theorem Explanation 1
• S={a b}
• S* = { all strings of a and b of any finite length }
• Concatenate strings from S* to form S**
• E.g., (aaba) (baaa) (aaba)
• aababaaaaaba
• = Concatenations of a and b, like the elements in S*
• ababaaaaaba
• = (aaba)(baaa)(aaba)
• = [(a)(a)(b)(a)][(b)(a)(a)(a)][(a)(a)(b)(a)]
• (a)(a)(b)(a)(b)(a)(a)(a)(a)(a)(b)(a)
Theorem Explanation 2
• S = {aa bbb}
• S* = {All strings where a's appear in even
clumps, and b's in groups of 3,6,9 ...}
• E.g., aabbbaaaa bbb bbbaa
• Concatenate these 3 to get one big word in S**
which is again, in S*
• aabbbaaaabbbbbbaa
• [(aa)(bbb)(aa)(aa)] [(bbb)] [(bbb)(aa)]
Questions
Chapter 2
• Q1, Q2, Q3,Q4,Q5 and

• Q) Let S = {a bb bab abaab}

▫ Is abbabaabab in S* (show by factorization)?
▫ Is abaabbabbaabb in S* (show by factorization)?
▫ Does any word in S* have an odd total number of
b’s (explain)?