S.NO.

CHAPTER PAGE

1. Introduction 2

2. Finite Automata 8

3. Regular Expression 14

4. CFG and PDA 17

5. Regular languages & 34

CFLs
6. Closure Properties & 36
Pumping Leema
7. Turing Machines 39

8. Undecidability 46
Introduction to Theory of Computation
In this subject , the only motive is to prove the following statement :

"Everything which has a logic can be simulated on a machine "

or, in other words , a problem which can be solved using an algorithm can also be solved by the
computer.

To understand this subject we will consider some machines like : Finite Automata , Push Down
Automata and Turing Machines. Using these models, we will solve all those problems which
have an algorithm behind them. For ex : all mathematical functions and logics. Multiplying two
numbers have logic behind them , so we can perform it using computer .

But suppose , I have 100 numbers on a screen , and I choose one number in my mind. The
computer cannot guess that number because this problem doesn’t have any logic behind it. I can
choose any  number I want.

This subject is divided into 3 main parts :

 Automata and languages

 Computability theory
 Time complexity

Automata: Study of abstract computing devices or machines.

Symbol: A symbol is an abstract entity i.e., letters and digits.

 Example: 0,1

Alphabet (∑): An alphabet is a finite, nonempty set of symbols.

 Example: Binary alphabet ∑ = {0, 1}

String: It is a sequence of symbols.

 Example: 0101 is a string

Finite String: It is a finite sequence of symbols.

  Example: 010 is a finite string which has length of 3.

Infinite String: It is an infinite sequence of symbols.

 Example: 011111… is an infinite string which has infinite length. (infinite strings are not
used in any formal language)

Language: A language is a collection of sentences of finite length all constructed from a finite
alphabet of symbols.

 Example: L = {00, 010, 00000, 110000} is a language over input alphabet ∑ = {0, 1}

Formal Language: It is a language where form of strings is restricted over given alphabet.

Example:

 Set of all strings where each string starts with 1 over binary alphabet.
 L={1, 10, 11, ...} over 0's and 1's.

Empty String (Λ or ε or λ): If length of the string is zero, such string is called as empty string
or void string.

Kleene Closure:

 If ∑ is the Alphabet, then there is a language in which any string of letters from ∑ is a
word, even the null string. We call this language closure of the alphabet.

 It is denoted by * (asterisk) after the name of the alphabet is ∑*. This notation is also
known as the Kleene Star.
 If ∑ = {a, b}, then ∑* = {ε, a, b, a, ab, bb,….}

∑* = ∑0 ∪ ∑1 ∪ ∑2 ∪ …

∑* = ∑+ ∪ {ε}

Positive Closure:
  The' +' (plus operation) is sometimes called positive Closure.
 If ∑ = {a}, then ∑+ = {a, aa, aaa, ...}  = the set of nonempty strings from ∑

∑+ = ∑* - {ε}

Concatenation of two strings:

 If x, y ∈ ∑*, then x concatenated with y is the word formed by the symbols of x followed
by the symbols of y.
 This is denoted by x.y, it is same as xy.

Substring of a string:

 A string v is a substring of a string ω if and only if there are some strings x and y such
that ω = xvy.

Suffix of a string:

 If ω = xv for some string x, then v is suffix of ω.

Prefix of a string:

 If ω = vy for some string y, then v is a prefix of ω.

Reversal of a string:

 Given a string ω, its reversal denoted by ωR is the string spelled backwards.

Grammar:

 It enumerates strings of the language. It is a finite set of rules defining a language.

 A grammar is defined as 4-tuples (VN, ∑, P, S),
  where, VN is finite non-empty set of non-terminals, ∑ is finite non-empty set of input
terminals, P is finite set of production rules, and S is the start symbol.

Chomsky Hierarchy: The Chomsky hierarchy consists of following four types of classes.

Type 0 Grammar (Unrestricted Grammar):

 These are unrestricted grammars which include all formal grammars.

 These grammars generate exactly all languages that can be recognized by a Turing
machine.
 Rules are of the form α → β,
 where, α and β are arbitrary sequence of terminals and non-terminals and α ≠ ∧ (null).

Type 1 Grammar (Context Sensitive Grammar):

 Languages defined by type-1 grammars are accepted by linear bounded automata.

  Rules are of the form X → Y, where, X, Y ∈(VN ∪ ∑)*, and Length of X is less than or
equal to length of Y.

Type 2 Grammar (Context-free Grammar):

 Languages defined by type-2 grammars are accepted by push-down automata.

 Rules are of the form A → α, where, A ∈VN , and α∈(VN ∪ ∑)*

Type 3 Grammar (Regular Grammar):

 Languages defined by type-3 grammars are accepted by finite state automata.

 Regular grammar can follow either right linear or left linear.
 Right linear rules are of the form: A → α | αB where, A, B ∈ VN and α∈∑*.
 Left linear rules are of the form: A → α | Bα where, A, B ∈ VN and α∈∑*.

Type-0 Class is also called as:

 Unrestricted Grammars
 Recursively Enumerable Languages
 Turing Machine

Type-1 Class is also called as:

 Context Sensitive Grammars

 Context Sensitive Languages
 Linear Bound Automata

Type-2 Class is also called as:

 Context Free Grammars

 Context Free Languages
 Push Down Automata

Type-3 Class is also called as:

  Regular Grammars
 Regular Languages
 Finite Automata

Finite Automata (FA):

 Machines with fixed amount of unstructured memory, accepts regular languages.

 Applications of FA: useful for modeling chips, communication protocols, adventure
games, some control systems, lexical analysis of compiler design, etc.

Pushdown Automata (PDA):

 Finite Automata with unbounded structured memory in the form of a pushdown stack,
accepts context free languages.
 Application of PDA: useful for modeling parsing, compilers, postfix evaluations, etc.

Turing Machine (TM):

 Finite Automata with unbounded tape, accepts or enumerates recursively enumerable

languages.
 Equivalent to RAMs, and various programming languages models.
 Applications of TM: Model for general sequential computation (real computer). 

Study Notes on Finite Automata

Finite Automaton

A Finite State Machine (FSM) or finite state automaton is an abstract machine used in the study of
computation and language that has only a finite, constant amount of memory.

Types of Finite Automaton:

 Deterministic Finite Automata (DFA)

 Non-Deterministic Finite Automata (NFA or NDFA)
 NFA with Epsilon moves (epsilon-NFA)

Description of Finite Automaton

A finite automaton is defined as 5-tuples (Q, Σ, δ, q 0, F). where, Q is finite non-empty set of states, Σ is
finite non-empty set of input d alphabets, δ is transition function which maps Q × Σ into Q, q 0 is initial
state and q0 ∈ Q, F is set of final states and F ⊆ Q.

 For DFA: Q × Σ → Q

 For NFA: Q × Σ → 2Q
 For epsilon-NFA : Q × Σ U {epsilon}→ 2Q
Transition Diagrams

A transition diagram is a finite directed labelled graph in which each vertex represents a state and directed
edges indicate the transition from one state to another. Edges are labelled with input/output. In this, the
initial state is represented by a circle with an arrow towards it, the final state is represented by two
concentric circles and intermediate states are represented by just a circle.

Example: Consider the following Finite Automata

Important observations for the above FA:

1. The finite automaton M1 has three states, labeled q1, q2, and q3; since these are the labels found in
the three circles.  Thus, Q = { q1, q2, q3 }

2. The input symbols for M1 are 0 and 1, as these are the only labels found on the arrows that
represent the transitions.  Thus,  Σ = { 0, 1 }.

3. The start state is q1, the state shown with an unlabeled input arrow coming from nowhere. 
4. The only state marked with the double circle is q3, so F = {q3}.

5. Transitions:

            d(q1, 0 ) = q2              d(q1, 1 ) = q1

            d(q2, 0 ) = q2              d(q2, 1 ) = q3
            d(q3, 0 ) = q2              d(q3, 1 ) = q1

6. FA accepts all strings where each string ends with 01.

Problem-1: Construct DFA that accepts the language L given below.

L = {x01y | x and y are any strings of 0’s and 1’s}.

DFA for above L:

 

In the given transition diagram, vertex A is the initial state of the finite automata and C is the final state.

Transition Table:

Transition table is the tabular representation of transition system.

In this representation, initial (start) state is represented by an arrow towards it and a final state is
represented by a circle or prefixed with star symbol.

Following transition table is equivalent to the above given transition diagram.

Acceptability of a String by Finite Automata

A string ω is accepted by a finite automaton, M = (Q, Σ, δ, q 0, F), if δ(q0, ω) = F, where q0 is initial (start)
state and F is the final state. For every input symbol of string it takes one transition and starts with initial
state and reaches final state by reading the complete string.
Transition Function

It takes two arguments i.e., a state and an input symbol. δ(q, a) is the transition function for the DFA
(Deterministic Finite Automata) which is at the state q and when it receives the input a, DFA will move to
the next state.

Extended Transition Function

It takes two arguments a state and an input string.

δ*(q, ω) = δ(δ(q, x), a)

where, ω is a string i.e., ω = xa, in which a is a single word and x is remaining string except the last
symbol.

Equivalence of DFA and NDFA

In contrast to the NFA (NDFA), the Deterministic Finite Automata (DFA) has

 No ε-transition (null transition).

 For every (q, a) with q ∈ Q and a ∈ Σ at most one successor state.
 Deterministic finite automata can simulate the behaviour of NFA by increasing the number of
states.
 In Deterministic Finite Automata (DFA), for each state, there is at most one transition for each
possible input.
 In non-deterministic finite automata, there can be more than one transition from a given state for
a given possible input.
 If a language L is accepted by an NFA, then there is a DFA that accepts L.
 All DFA are NDFA, but not all NFA are DFA.
 All NDFA and DFA have the same power.
 Processing of an input string is more time consuming when NFA is used and less time consuming
when DFA is used.
Finite Automata with Output Capabilities

There are two types of machines: Moore machine and Mealy machine.

Moore Machine

 It is a finite automata in which output is associated with each state.

 Moore machine is defined as 6-tuples (Q, Σ, Δ, δ, λ, q0)

where, Q is finite non-empty set of states.

 Σ is set of input symbols.

 Δ is output alphabet.
 δ is transition function which maps δ(Σ × Q) → Q.
 λ is output function which maps Q into Δ.
 q0 is initial state.

Mealy Machine

It is a finite automata in which the output depends upon both the present input and present state. Mealy
machine is also a 6-tuple (Q, Σ, Δ, δ, λ, q 0), where all symbols except λ have the same meaning as in
Moore machine. λ is the output function mapping Σ × Q into Δ.
Sequence detector 11011 using Mealy machine:
Study Notes on Regular Expressions
Regular Expression

Regular expressions mean to represent certain sets of strings in some algebraic fashion. A regular
expression over the alphabet Σ is defined as follows

 ϕ is a regular expression corresponding to the empty language ϕ.

 ε is a regular expression corresponding to the language { ε }.
 For each symbol a ∈Σ a is a regular expression corresponding to the language {a}.

Regular Language

The languages accepted by FA are regular languages and these languages are easily described by simple
expressions called regular expressions.

For any regular expression r and s over Σ corresponding to the languages L r and Ls respectively, each of
the following is a regular expression corresponding to the language indicated.

 (rs) corresponding to the language LrLs

 (r + s) corresponding to the language Lr ∪ Ls
 r* corresponding to the language Lr. Some examples of regular expression are
1. L (01) = {0, 1}
2. L (01 + 0) = {01, 0}
3. L (0 (1+ 0)) = {01, 00}
4. L (0*) = {ε, 0, 00, 000, ...}
5. L ((0 + 10)* (ε + 1)) = all strings of 0's and 1's without two consecutive 1's.
 If L1 and L2 are regular languages in Σ*, then L1 ∪ L2, L1 ∩ L2, L1 – L2 and L1 (complement of L1),
are all regular languages.
 Pumping lemma is a useful tool to prove that a certain language is not regular.
Regular Set

A set represented by a regular expression is called regular set e.g., If Σ = {a, b} is an alphabet, then

Different Regular Sets and their Expressions

Identities for Regular Expressions 

The following points are the some identities for regular expressions.

 ϕ + R = R + ϕ = R
 εR=Rε=R
 R + R = R, where R is the regular expression.
 (R*)* = R* 
 ϕR = Rϕ = ϕ
 ε * = ε and ϕ* = ε
 RR* = R*R = R+
 R*R* = R*
 (P + Q)* = (P*Q*)* = (P* + Q*)*, where P and Q are regular expressions.
 R (P + Q) = RP + RQ and (P + Q)R = PR + QR
 P(QP)* = (PQ)*P 
 Arden’s Theorem 
 If P and Q are two expressions over an alphabet Σ such that P does not contain ε, then the
following equation R = Q + RP.

 The above equation has a unique solution i.e., R = QP *. Arden's theorem is used to determine the
regular expression represented by a transition diagram.

 The following points are assumed regarding transition diagrams

 The transition system does not have any

 It has only one initial (starting) stage.

Properties of Regular Language

Regular languages are closed under following opearations

1. Union
2. Concatenation
3. Kleene closure
4. Complementation
5. Transpose
6. Intersection
Study Notes on Context-Free Grammars and
Push-Down Automata
Context Free Language

 The languages which are generated by context-free grammars are called Context-Free Languages
(CFLs). 
 CFLs are accepted by Push down Automata.
 CFLs are also called as non-deterministic CFL.

Definition: If v is one-step derivable from u, written u ⇒ v. If v is derivable from u, written u  if

there is a chain of one derivation of the form u ⇒ u1 ⇒ u2 ⇒ … ⇒ v

Example: Consider the context free grammar G = ({s}, {0, 1}, P, S) where Productions are:

(i) S → 0S1

(ii) S →ε

Derivations are:

 
Derivations:

A derivation of a string is a sequence of rule applications. The language defined by a context-

free grammar is the set of strings derivable from the start symbol S (for sentence).

A string can be derived with any of the following derivations.

Left Most Derivation:

 In each sentential form, left most non-terminal substituted first to derive a string from the starting
symbol.
 A derivation is left most if at each step in the derivation a production is applied to the left most
non-terminal in the sentential form.

Right Most Derivation: 

 In each sentential form, right most non-terminal substituted first to derive a string from the
starting symbol.
 A derivation is left most if at each step in the derivation a production is applied to the left most
non-terminal in the sentential form.

Example:

 Every derivation corresponds to one derivation tree.

S ⇒ AB

⇒ aAAB

⇒ aaAB

⇒ aaaB

⇒ aaab

 Every derivation tree corresponds to one or more derivations.

S ⇒ AB S ⇒ AB S ⇒ AB

⇒ aAAB ⇒ Ab ⇒ AB

⇒ aaAB ⇒ aAAb ⇒ aAAb

⇒ aaaB ⇒ aAab ⇒ aaAb

⇒ aaab ⇒ aaab ⇒ aaab

Derivation Tree (Parse Tree)

 A derivation tree (or parse tree) can be defined with any non-terminal as the root, internal nodes
are non-terminals and leaf nodes are terminals.
 Every derivation corresponds to one derivation tree.
 If a vertex A has k children with labels A1, A2, A3,…Ak, then A → A1 A2 A3…Ak will be a
production in context-free grammar G.

Example:  

S → AB, A → aAA, A → aA, B → bB, B → b

  

Ambiguous Grammar
A context-free grammar G is ambiguous if there is atleast one string in L(G) having two or more distinct
derivation trees (or equivalently, two or more distinct left most derivations or two or more distinct right
most derivations).

e.g., consider the context-free grammar G having productions E → E + E/a. The string a + a + a has two
left most derivations.

Let's see the derivations

E ⇒ E + E ⇒ E + E + E ⇒ a + E + E ⇒ a + a + E ⇒ a + a + a

E ⇒ E + E ⇒ a + E ⇒ a ⇒ E + E ⇒ a + a + E ⇒ a + a + a

and the derivation trees are

CFG Simplification

The four main steps will be followed in CFG simplification

 Eliminate ambiguity.
 Eliminate useless symbols productions.
 Eliminate ∧ productions: A → ∧
 • Eliminate unit productions: A → B

Eliminate the Ambiguity

We can remove the ambiguity by removing the left recursing and left factoring.

Left Recursion

A production of the context free grammar G = (VN, E, P, S) is said to be left recursive if it is of the form
A → Aα

Where A is a nonterminal and

α∈(VN ∪ E)*

Removal of Left Recursion

Let the variable A has left recursive productions as follows

(i) A → Aα1 |Aα2|Aα3|…|Aαn|β1|β2|β3|…|βn

Where β1, β2 ..... βn do not begin with A. Then we replace A production in the form of

(ii) A → β1A1 |β2A1|…|βmA1 where

A1 → α1A1|α2A1|α3A1|…,|αnA1|∧

Left Factoring

Two or more productions of a variable A of the grammar G = (V N, E, S, P) are said to have left factoring
if the productions are of the form

A → αβ1|αβ2|…αβn where β1,…βn(VN ∪Σ)

Removal of Left Factoring

Let the variable A has left factoring productions as follows

A → αβ1|αβ2|…|αβn|y1|y2|y3|…|ym

where, β1, β2…..,βn have a common factor

α and y1, y2,….ym does not contain a as a prefix, then we replace the production into the form as follows

A → αA1|Y1Y2|…..|YM, where
A1 → β1|β2|…..|βn

Eliminate the Useless Productions/Symbols

The symbols that cannot be used in any productions due to their unavailability in the productions or
inability in deriving the terminals are known as useless symbols.

e,g., consider the grammar G with the following production rules

S → aS |A| C

A → a

B → aa

C → ab

Step 1 Generate the list of variables those produce terminal symbols

U = {A, B, S}

Because C does not produce terminal symbols so this production will be deleted. Now the modified
productions are

S → aS |A

A → a

B → aa

Step 2 Identify the variables dependency graph

S → AB

In this graph, B variable is not reachable from S so it will be deleted also. Now the productions are
S → aS |A

A → a

Eliminate Null Productions

If any variable goes to ∧ then that is called as a nullable variable.

e.g., A → ∧, then variable A is said to be nullable variable

Step 1 Scan the nullable variables in the given production list.

Step 2 Find all productions which do not include null productions.

(Null productions)

e.g., consider the CFG has following productions

S → ABaC

A → BC

B → b|∧

C → D|∧

D → d

solve step find the nullable variables Firstly the set is empty

N = {}

N = {B, C}

N = {A, B, C}
Due to B, C variables, A will also be a nullable variable.

Step 3   {Null productions}

S → BaC | AaC | ABa | aC | Ba | Aa | a

A → B | C

B → b

C → D

D → d

The above grammar is the every possible combination except ∧ Now put this new grammar with original
grammar with null.

S → ABaC | BaC | AaC | ABa | aC | Ba | Aa | a

Eliminate the Unit-Productions

A production of the type A → B, where A, B are variables is called unit productions.

Step 1 Using productions, we create dependency graph

S ⇒ B

∵ S   B & B   A

∴ S   A

Step 2 Now the production without unit productions

S → Aa S → bb | a | bc

B → bb + A → bb

A → a | bc B → a | bc

Now the final grammar is

S → Aa | bb | a | bc

B → bb | a | bc

A → a | bc | bb

Normal Forms of CFGs

Ambiguity is the undesirable property of a context-free grammar that we might wish to eliminate. To
convert a context-free grammar into normal form, we start by trying to eliminate null productions of the
form A → ∧ and the unit productions of the form B → C.

There are two normal forms

1. Chomsky Normal Form (CNF)

2. Greibach Normal Form (GNF)
Chomsky Normal Form (CNF)

A context-free grammar G is said to be in Chomsky Normal Form, if every production is of the form
either A → a, (exactly a single terminal in the right hand side of the production) or A → BC (exactly two
variables on the right-hand side of the production).

e.g., the context-free grammar G with productions S → AB, A → a, B → b is in Chomsky normal form.

Chomsky Normal Form Properties

 The number of steps in the derivation of any string ω of length n is 2n – 1, where the grammar
should be in CNF.
 The minimum height of derivation tree of any ω of length n is [log 2 n] + 1.
 The maximum height of derivation tree of any ω of length n = n.

Greibach Normal Form (GNF)

A context-free grammar is said to be in Greibach Normal Form if every production is of the form

A → aα

where, a∈Σ, A∈VN and α∈

Deterministic Context-Free Language (DCFL)

The set of deterministic context-free languages is a proper subset of the set of context-free languages that
possess an unambiguous context-free grammar.
Key Points

 The problem of whether a given context-free language is deterministic is undividable.

 Deterministic context-free languages can be recognized by a deterministic turning machine in
polynomial time and O (log2 n) space.
 The language of this class has great practical importance in computer science as they can be
passed much more efficiently than non-deterministic context-free languages.

Pushdown Automata (PDA)

A Pushdown Automata (PDA) is essentially an NFA with a stack. A PDA is inherently non-deterministic.
To handle a language like {an bn |n ≥ 0}, the machine needs to remember the number of a's and b's. To do
this, we use a stack. So, a PDA is a finite automaton with a stack. A stack is a data structure that can
contain a number of elements but for which only the top element may be accessed.
Definition of PDA

A Pushdown Automaton (PDA) is defined as 7-tuple.

M = (Q, Σ, Γ, δ, q0, Z,F)

where Q is a finite set of states

Σ is the input alphabet

Γ is the stack alphabet

δ is the transition function which maps

(Q × (Σ ∪ {ε}) × (Γ ∪{ε}) = (Q × (Γ ∪ {ε}))

q0 is the start state and ε denotes the empty string.

q0∈Q is start state

Z ∈Γ is the initial stack symbol

F ⊆ Q is the set of final or accepting states.

Acceptance of PDA

Tape is divided into finitely many cells. Each cell contains a symbol in an alphabet L. The stack head
always scans the top symbol of the stack as shown in figure.

It performs two basic operations

Push add a new symbol at the top.

Pop read and remove the top symbol.

 δ (q, a, v) = (p, u)

It means that if the tape head reads input a, the stack head read v and the finite control is in state q, then
one of the possible moves is that the next state is p, v is replaced by u at stack and the tape head moves
one cell to the right.

δ (q, ε, v) = (p, u)

It means that this is a ε -move (null move)

δ (q, a, ε) = (p, u)
It means that a push operation performs on stack.

δ (q, a, v) = (p, ε)

It means that a pop operation performs on stack.

PDA can accept a string in three ways:

 PDA acceptance by Empty Stack: If the stack is empty after reading the entire input string then
PDA accepted the given string, otherwise rejected.
 PDA acceptance by Final State: If the stack reaches final state after reading the input string then
PDA accepted the given string, otherwise rejected.
 PDA acceptance by Final State and Empty Stack: If the stack reaches final state and also stack
is empty after reading the entire input string then PDA accepted the given string, otherwise
rejected.

Non-deterministic PDA: Like NFA, Non-deterministic PDA (NPDA) has a number of choices for its
inputs. An NPDA accepts an input, if sequence of choices leads to some final state or causes PDA to
empty its stack.

Deterministic PDA

Deterministic PDA (DPDA) is a pushdown automata whose action is a situation is fully determined rather
than facing a choice between multiple alternative actions. DPDAs cannot handle languages or grammars
with ambiguity. A deterministic context-free language is a language recognised by some deterministic
pushdown automata.

Following languages are DCFLs

 L = {anbn : n ≥ 0}
 L = {ancb2n : n ≥ 0}
 L = {ωcωR : ω∈(a + b) * but not L = {ωωR : (a + b)*}
 For every regular set, there exists a CFG e such that L = L (G).
  Every regular language is a CFL.
 Let G1 and G2 be context-free grammars. Then, G1 and G2 are equivalent if and only if L (G1) = L
(G2).

 The intersection of a context-free language and a regular language is a context-free language.

 The reverse of a context-free language is context-free.
 A DFA can remember only a finite amount of information whereas a PDA can remember an
infinite amount of information.
 For every PDA, there is a context-free grammar and for every context-free grammar, there is a
PDA.
 If L1 is a DCFL and L2 is regular then, L1 ∪ L2 is also DCFL.
 If L1 is a DCFL and L2 is a regular language, then L1 ∩ L2 is also DCFL.
 Every regular language is DCFL.
 The power of nondeterministic pushdown automata and deterministic pushdown automata is not
same. But the power of nondeterministic pushdown automata and deterministic pushdown is
same.
 An FSM (Finite State Machine) with one stack is more powerful than FSM without a stack.
 If left recursion or left factoring is present, it is not sure that the grammar is ambiguous but there
may be a chance of ambiguity.

Closure Properties of CFLs

CFLs are closed under following properties:

 Union
 Concatenation
 Kleene closure
  Positive closure
 Substitution
 Homomorphism
 Inverse homomorphism
 Reversal
 Intersection with regular
 Union with regular
 Difference with regular

CFLs are not closed under following properties :

 Intersection
 Complementation
 Difference

Decision Properties of CFL’s

Decidable Problems:

 Emptiness Problem: Is a given L(G) accepts empty?

 Membership problem: Is a string w accepted by given PDA?
 Finiteness Problem: Is a given L(G) accepts finite language?

Undecidable Problems:

 Equivalence problem: Are two CFLs the same?

 Ambiguity problem: Is a given CFG ambiguous?
 Is a given CFL inherently ambiguous?
 Is the intersection of two CFLs empty?
 Totality problem: Is a given L(G) equal to ∑*?
Study Notes on Regular and Context Free
Languages
1. Every regular language is also CFL, but every CFL need not be regular.
2. Every DCFL is also CFL, but every DCFL need not be regular.
3. Every regular language is also DFCL, but every DCFL need not regular.

Regular Languages

1. {w | w ∈ {a, b }* }
2. {aw | w ∈ {a, b }* }
3. {bw | w ∈ {a, b }* }
4. {wa | w ∈ {a, b }* }
5. {awb | w ∈ {a, b }* }
6. {w1abw2 | w1,w2 ∈ {a, b }* }
7. { ambn | m,n>0 }
8. { ambnck | m,n,k>=0}
9. {a2n | n>=0}

Non-Regular Languages

1. { anbn | n is a positive integer }

2. { ww | w ∈ {a, b }* }
3. {w | w has an equal number of a’s and b’s}
4. {w| w is a palindrome of a’s and b’s}
5. {an | n is prime}
Deterministic CFLs (DCFLs)

1. { anbn | n is a positive integer }

2. {w | w has an equal number of a’s and b’s}
3. { ambn | m < n}
4. { ambn | m = 2n}
5. { ambnck | if m is even, then n=k}
6. {wCwR  | w ∈ {a, b }* , C is a special symbol and wR is the reverse of string w}

CFL’s (NCFL’s)

1. {wwR  | w ∈ {a, b }* , and wR is the reverse of string w}

2. { ambnck | m=n or n=k}
3. { ambnck | if m=n, then n=k}
4. All regulars
5. All DCFLs

Non-CFL’s

1. { ww  | w ∈ {a, b }*}

2. { anbncn | n>=0}
3. {an | n is prime}
4. { ambnck | m<n<k}

Important Properties:

1. Let L be a Context Free Languages, and R be a regular language. Then

1. L ∩ R = always CFL and need not be regular
2. L ∪ R = always CFL and need not be regular
3. L - R  = always CFL and need not be regular
4. R - L = Always CSL but need not be CFL
2. Let D be a DCFL, and R be a regular language. Then
1. D ∩ R = always DCFL and need not be regular
2. D ∪ R = always DCFL and need not be regular
3. D - R  = always DCFL and need not be regular
4. R - D = Always DCFL but need not be regular
Study Notes on Closure properties of
Languages
The table below shows the closure properties of all the formal languages. 

Note :Union , Intersection or difference with regular doesn't change the language. i.e.

Let (AnyLANG) below represents any of the language among DCFl , CFl , CSL , RE or REC
Regular ∩ AnyLANG = AnyLANG

Regular ∪ AnyLANG = AnyLANG

Regular - AnyLANG = AnyLANG 

Study Notes of Pumping Lemma

Pumping Lemma for regular languages

 Suppose that a language L is regular. Then there is a FA that accepts L.

 Let n be the number of states of that FA. Then for any string x in L with |x| ≥ n, there are
strings u, v and w which satisfy the following:
o x = uvw
o |uv| ≤ n
o |v| > 0 is same as v ≠ ε
o For every integer m ≥ 0, uvmw ∈ L.
 If L is regular then for every x such that |x| ≥ n then there exists uvw such that x=uvw, v ≠
ε, |uv| ≤ n, and for which uviw is in L for every i.

Pumping Lemma gives a necessity for regular languages.

Pumping Lemma is not a sufficiency, that is, even if there is an integer n that satisfies the
conditions of Pumping Lemma, the language is not necessarily regular.

Pumping Lemma can not be used to prove the regularity of a language.

It can only show that a language is non-regular.

Example: L = akbk is non-regular, where k is a natural number.

 Suppose that L is regular and let n be the number of states of an FA that accepts L.
Consider a string x = anbn for that n.
 Then there must be strings u, v, and w such that
 x = uvw, |uv| ≤ n |v| > 0, and for every m ≥ 0, uvmw ∈ L.
 Since |v| > 0, v has at least one symbol.
  Also since |uv| ≤ n, v = ap, for some p > 0,
 Let us now consider the string uvmw for m = 2.
 Then uv2w = an-pa2pbn = an+pbn. Since p > 0 , n + p ≠ n .
 Hence an+pbn can not be in the language L represented by akbk.
 This violates the condition that for every m ≥ 0, uvmw ∈ L.
 Hence L is not a regular language.

Pumping Lemma for CFL’s

 Let L be a CFL. Then there exists a constant N such that if z ∈L s.t. |z|≥N, then we can
write z=uvwxy,
 |vwx| ≤ N
 vx ≠ ε
 For all k ≥ 0 : uvkwxky ∈ L
Study Notes on Turing Machines
Turing Machine

The languages accepted by Turing machine are said to be recursively enumerable. A Turing
Machine (TM) is a device with a finite amount of read only hard memory (states) and an
unbounded amount of read/write tape memory.

 Recursive languages are closed under complementation, union, intersection,

concatenation and Kleene closure.
 A Turing machine is said to be partially decide a problem, if the following two conditions
are satisfied.
1. The problem is a decision problem.
2. The Turing machine accepts as given input if and only if the problem has an
answer 'yes' for the input that is the Turing machine accepts the language L.
 A Turing machine is said to be decide a problem, if it partially decides the problem and
all its computations are halting computations.

A language L is Turing-recognisable if there is a Turing machine M such that L=L(M).

A language L is Turing-decidable if there is a Turing machine M such that M decides L.

Turing-decidable language is also Turing-recognisable, but Turing-recognisable may not be

Turing-decidable.

A language is recursively enumerable iff it is Turing-enumerable.

Universal Turing Machine (UTM)

  A UTM is a specified Turing machine that can simulate the behaviour of any TM.
 A UTM is capable of running any algorithm.

For simulating even a simple behaviour, a Universal Turing Machine must have a large number
of states. If we modify our basic model by increasing the number of read/write heads, the number
of dimensions of input tape and adding a special purpose memory, then we can design a
Universal Turing Machine. 

Definition of Turing Machine

A Turing Machine (TM) is defined as 7-tuples.

TM = (Q, Σ, Γ, δ, q0, b, F), 

where, Q is a finite non-empty set of states, Σ is a non-empty set of input symbols (alphabets)
which is a subset of Γ and b ∈ Σ, Γ is a finite non-empty set of tape symbols, δ is the transition
function which maps (Q × Γ) to (Q × Γ × {L, R}), q0 is the initial state and q0 ∈ Q, b is the blank
and b ∈ Γ, F is the set of final states and F ⊆ Q.

Transition Function of a Turing Machine

The transition function Q × Γ → Q × Γ × {L, R} states that if a Turing machine is in some state
(from set Q), by taking a tape symbol (from set Γ), it goes to some next state (from set ï) by
overwriting (replacing) the current symbol by another or same symbol and the read/write head
moves one cell either left (L) or right (R) along the tape.
 

  

 
3. Construct a TM that accepts the language A = {0(2^n) | n>=0}

Behaviour of Turing Machine

Depending upon the number of moves in transition, a TM may be deterministic or non-

deterministic. If TM has at most one move in a transition, then it is called Deterministic TM
(DTM), if one or more than one move, then Non-deterministic TM (NTM or NDTM).

 A non-deterministic TM is equivalent to a deterministic TM.

 Some single tape TM simulates every 2 PDA (a PDA with two stacks).
 The read only TM may be considered as a Finite Automata (FA) with additional property
of being able to move its head in both directions (left and right).
 

Language Recognition by Turing Machine

TM can be used as a language recogniser. TM recognises all languages, regular language, CFL,
CSL, Type-0.

There are several ways an input string might fail to be accepted by a Turing machine

 It can lead to some non-halting configuration from which the Turing machine cannot
move.
 At some point in the processing of the string, the tape head in scanning the first cell and
the next move specifies moving the head left off the end of the tape.
 In either of these cases, we say that the Turing machine crashes

Variation of TM with other Automata

 Multitape Turing Machine A Turing machine with several tapes is said to be a

multitape Turning machine. In a multitape Turing machine, each tape is controlled by its
own independent read/write head.
 Turing machine with multiple tape is no more powerful that one tape Turing machine.
 Multi-dimensional Turing Machine A Turing machine is said to be multi-dimensional
Turing machine, if its tape can be viewed as extending infinitely in more than one
dimension.
 Multihead Turing Machine A multihead Turing machine can be viewed as a Turing
machine with a single tape and a single finite state control but with multiple independent
read/write heads.
 In one move, the read/write heads may take move independently left, right or remain
stationary
  Offline Turing Machine An offline Turing machine is a multitape Turing machine
whose input tape is read only (writing is not allowed). An offline Turing machine can
simulate any Turing machine A by using one more tape than Turing machine A. The
reason of using an extra tape is that the offline Turing machine makes a copy of its own
input into the extra tape and it then simulate Turing machine A as if the extra tape were
A's input.

Halting Problem of Turing Machine: A class of problems with two output (true/false) is called
solvable (or decidable) problem, if there exists some definite algorithm which always halts (also
called terminates), else the class of problem is called unsolvable (or undecidable.

Recursive and Recursively Enumerable Languages

A language L is said to be recursively enumerable, if there exists a Turing machine that accepts
it.

A language is recursive if and only if there exists a membership algorithm for it. Therefore, a
language L on Σ is said to be recursive, if there exists a Turing machine that accepts the language
L and 'it halts on every ω∈Σ+.

Recursively enumerable languages are closed under union, intersection, concatenation and
Kleene closure and these languages are not closed under complementation.

 The complement of a recursive language is recursive.

 The union of two recursive languages is recursive.
 The union of two recursiv enumerable languages is recursive enumerable.
 Intersection of two recursive languages isrecursive.
 There are some recursively enumerable languages which are not recursive.
 If L is recursive then, L' is also recursive and consequently both languages are recursively
enumerable.
 A language is recursive iff both it and its complement are recursively enumerable.
 A language L is lexicographically Turing-enumerable iff there is a Turing machine that
lexicographically enumerates it.
 A language is recursive iff it is lexicographically Turing-enumerable.
 Every context sensitive language is recursive.
 The family of recursively enumerable languages is closed under union.
 If a language is not recursively enumerable, then its complements cannot be recursive.
  If a languages L is recursive, then it is recursively enumerable language but vice-versa is
not true.

An infinite set is countable if and only if there is a one-to-one correspondence between its
elements and the natural numbers. Otherwise it is said to be uncountable.

 If Σ is a finite set then Σ ∗ is countable.

 For every alphabet Σ there is a language L ⊆ Σ ∗ that is not recursively enumerable.
 There exists a language L that is recursively enumerable but not decidable.
 The halting problem is the problem of deciding whether a given Turing machine halts
when presented with a given input. The halting problem is not decidable.

Study Notes on Undecidability

There are two types of TMs (based on halting):

1. Halting TM : (Accepts Recursive languages) : TMs that always halt, no matter accepting
or non no matter accepting or non-accepting (called as decidable problems)

2. TM : (Accepts Recursively enumerable): TMs that are guaranteed to halt are guaranteed
to halt only on acceptance only on acceptance. If non-accepting, it may or may not halt
(i.e., could loop forever). (Either decidable or partially decidable)

Decidable Problem

 If there is a Turing machine that decides the problem, called as Decidable problem.
 A decision problem that can be solved by an algorithm that halts on all inputs in a finite number
of steps.
 A problem is decidable, if there is an algorithm that can answer either yes or no.
 A language for which membership can be decided by an algorithm that halts on all inputs in a
finite number of steps.
 Decidable problem is also called as totally decidable problem, algorithmically solvable,
recursively solvable.

Undecidable Problem (Semi-dedidable or Totally not decidable)

 A problem that cannot be solved for all cases by any algorithm whatsoever.

 Equivalent Language cannot be recognized by a Turing machine that halts for all inputs.

The following problems are undecidable problems:

 Halting Problem: A halting problem is undecidable problem. There is no general method or

algorithm which can solve the halting problem for all possible inputs.
 Emptyness Problem: Whether a given TM accepts Empty?
 Finiteness Problem: Whether a given TM accepts Finite?
 Equivalence Problem: Whether Given two TM’s produce same language?. Is L(TM1) =
L(TM2) ?
 Is L(TM1) ⊆ L(TM2) ? (Subset Problem)
 Is L(TM1) Ո L(TM2) = CFL?
 Is L(TM1) = Σ* ? (Totality Problem)
 Is the complement of L(G1) context-free ?

Undecidable problems are two types: Partially decidable (Semi-decidable) and Totally not decidable.
  Semi decidable: A problem is semi-decidable if there is an algorithm that says yes. if the answer
is yes, however it may loop infinitely if the answer is no.
 Totally not decidable (Not partially decidable): A problem is not decidable if we can prove that
there is no algorithm that will deliver an answer.

 
 
  Decidability table for Formal Languages:
Problems RL DC CFL Rec RE
Membership Y Y Y Y N
Finiteness Y Y Y N N
Emptiness Y Y Y N N
Equivalence Y Y N N N
Is L1 ⊆ L2 ?(SUBSET) Y N N N N
Is L = REGULAR? Y Y N N N
Is L Ambiguous? Y N N N N
L=∑* ?(UNIVERSAL) Y Y N N N
L1 ∩ L2= Ф ?(DISJOINT) Y N N N N
Is L= Regular? Y Y N N N
L1 ∩ L2= L Y N N Y Y
Is L' also same type? Y Y N Y N

 Notes : RL= Regular Language , DC = Deterministic context-free languages (DCFL), CFL=

Context Free Languages (CFL), Rec =  Recursive language, RE= Recusively Enumerable
Language.