20CS5101 - THEORY

OF COMPUTATION
SYLLABUS

OBJECTIVES

 To understand the relation between

these formal languages, grammars, and
machines.
SYLLABUS
 COURSE OUTCOMES

 Students will be able to

 1. Explain proofing techniques and construct
finite automata
 2. Generate finite automata for regular
expression using its properties
 3. Apply context-free grammar and languages
 4. Construct Pushdown Automata and Turing
machine
 5. Analyze the undecidability of languages.
SYLLABUS
 UNIT 1 - FINITE AUTOMATA

 Introduction to formal proof - Additional

forms of proof - Inductive proofs - Finite
Automata (FA) Deterministic Finite
Automata (DFA) - Non-deterministic Finite
Automata (NFA) - Finite Automata with
Epsilon transitions-Equivalence of NFA
and DFA-Equivalence of NFAs with and
without Epsilon
SYLLABUS
 UNIT 2 - REGULAR EXPRESSIONS
AND LANGUAGES

 Regular Expression - FA and Regular

Expressions - Arden's theorem -
Applications of Regular Expression -
Algebraic Laws for Regular Expression -
Proving languages not to be regular -
Closure properties of regular languages.
SYLLABUS
 UNIT 3 – CONTEXT-FREE GRAMMAR
AND LANGUAGES

 Grammar Introduction- Types of Grammar

- Context-Free Grammar (CFG) - Parse
Trees - Applications of Context-Free
Grammar -Ambiguity in grammars and
languages - Normal forms for CFG –CNF-
GNF- Pumping Lemma for CFL - Closure
Properties of CFL.
SYLLABUS
 UNIT 4 - PUSHDOWN AUTOMATA AND
TURING MACHINES

 Definition of the Pushdown automata -

Languages of a Pushdown Automata -
Equivalence of Pushdown automata and
CFG. Turing Machines (TM)- Problems
about Turing machine -Programming
Techniques for TM –Chomskian hierarchy
of languages
SYLLABUS
 UNIT 5 – UNDECIDABILITY

 A language that is not Recursively

Enumerable (RE) - An Undecidable
problem that is RE - Undecidable
problems about Turing Machine - Post's
Correspondence Problem - Rice
Theorem-P and NP problems
SYLLABUS
 TEXT BOOKS


 1. J.E.Hopcroft, R.Motwani and J.D Ullman, “Introduction to Automata

Theory, Languages and Computations”, Pearson Education, Third Edition,
2011.
 REFERENCE BOOKS


 1. Mishra K L P and Chandrasekaran N, “Theory of Computer Science-

Automata, Languages and
 Computation”, Prentice Hall of India, Third Edition, 2007.
 2.Harry R. Lewis and Christos H. Papadimitriou, “Elements of the theory of
Computation”, Prentice-Hall of India Pvt. Ltd, Second Edition, 2009.
 3.J. Martin, “Introduction to Languages and the Theory of Computation”,
Tata Mc Graw Hill, New Delhi,
 Third Edition, 2007.
 4.MichealSipser, “Introduction to the Theory and Computation”, Cengage
Learning India, 2012.
 INTRODUCTION

THEORY OF COMPUTATION
 The Theory of Computation is a

branch of computer science.

 It deals with how efficiently

problems can be solved on a

model of computation using an
algorithm.
 It is studying the nature of

computation to understand and

INTRODUCTION

YES

Input
Think

NO
INTRODUCTION

Undecidable

Turing
machine
CFL
FS
M
Need for Automata Theory

 The word computability is for the

computation in mathematical modeling.
 This course deals with the study of
various mathematical models that are
required for some computations.
 The theory of computation provides a set
of abstract structures that are useful for
solving certain classes of problems.
Need for Automata Theory

 Using these abstract structures the

required design efforts can be
determined for the actual model.
 Theory of computation plays an
important role in compiler design.
 Many times to prove the correctness of
the program, automata theory is used.
Introduction to Formal Proof

 The formal proof can be using deductive

proof and inductive proof.
 The deductive proof consists of sequence of
statements given with logical reasoning in
order to prove the first or initial statement.
The initial statement is called hypothesis.
 The inductive proof is a recursive kind of
proof which consists of sequence of
parameterized statements that use the
statement itself with lower values of its
parameter.
Deductive Proof
 A deductive proof consists of a sequence of
statements whose truth leads us from some
initial statement called the hypothesis or the
given statement(s) to a conclusion statement
.
 The theorem that is proved when we go from
a hypothesis H to a conclusion C is the
statement “if H then C”. We say that C is
deduced from H.
Deductive Proof
Theorem - If X ≥ 4 then 2X≥X2
Proof:
 First notice hypothesis H is “x ≥ 4 ”

 Hypothesis has parameter x and thus

neither true or false. It is truth depends on

the value of the x.
 Ex. X=6 is true and false for x=2.

 Like wise Conclusion C is “2 X≥X2”.

 C is also use parameter x and it is true for

certain values and false for certain values.

Deductive Proof
Theorem - If X ≥ 4 then 2X≥X2
Proof:
 If x = 3 then 23 =8, 32 = 9, C= 8 ≥ 9 is false

 If x = 4 then 24 =16, 42 = 16, C= 16 ≥ 16 is true

 If x = 5 then 25 =32, 52 = 25, C= 32 ≥ 25 is true

 The intuitive argument that tells us the

conclusion
2X≥X2 will be true whenever X ≥ 4
Deductive Proof
Theorem - If x is the sum of the squares of
four positive integers then 2X≥X2
Proof:
Reduction to definitions
 A set S is finite if there exists an integer
n such that S has exactly n elements. We
write ||S||= n where ||S|| is used to
denote the number of elements in a set
S.
 If S and T are both subsets of some set U
then T is the complement of S (with
respect to U ) if S∪T = U and S ∩ T= Ø.
Reduction to definitions
Theorem
 Let S be a finite subset of some infinite

set U. Let T be the complement of S with

respect to U. Then T is infinite.
 Proof :
Reduction to definitions
 We know that S ∪ T= U and S and T are
disjoint.
 So ||S||+||T|| = ||U||.
 Since S is finite ||S||=n for some integer n
 since U is infinite there is no integer p such
that ||U || = p
 So assume that T is finite
that is ||T||= m for some integer m.
 Then ||U||=||S||+||T|| = n+m which
contradicts the given statement that there is
no integer p equal to ||U||.
Other theorem forms
Ways of Saying “If Then”
 H implies C

 H only if C

 C if H

 Whenever H holds C follows

Additional Forms of Proof

 1. Proofs about sets.

 2. Proofs by contradiction.
 3. Proofs by counter example.
1. Proofs about sets.
 The set is a collection of elements or
items. By giving proofs about the sets
we try to prove certain properties of the
sets.
 For Example, the kind "if and only if"
that means an element x is in A if and
only if it is in B.
1. Proofs about sets.
Theorem :
 Let us prove PUQ = QUP.
1. Proofs about sets.
Hence PUQ = QUP. Thus A = B is true as
element x is in B if and only if x is in A.
Theorem :
R U (S ∩ T) = (R U S) ∩(R U T)
PROOF :
The two set expressions involved are E =
R U (S ∩ T) and F = (R U S) ∩(R U T)
In the “if” part we assume element x is in
E and show it is in F
1. Proofs about sets.
1. Proofs about sets.
2. Proofs by Contradiction .

 The contrapositive of the statement “if H

then C” is “if not C then not H”
 “if H then C” and “if not C then not H”
are logically equivalent.
2. Proofs by Contradiction .

 The contrapositive of the statement “if H then C”

is “if not C then not H”
 “if H then C” and “if not C then not H” are
logically equivalent.
 Example:
 Prove PUQ = QUP.
 Proof
 Initially we assume that PUQ = QUP is not true.
That is PUQ ≠ QUP.
 Now consider that x is in Q, or x is in P. Hence we
can say x is in PUQ (according to definition of
union).
2. Proofs by Contradiction .

 But this also implies that x is in QUP

according to definition of union.
 Hence the assumption which we made
initially is false.
 Thus PUQ = QUP is proved.
2. Proofs by Contradiction .

 Example :
 Prove that √2 is not rational.
 Proof: We will assume that, √2 is a
rational number.
 √2 = a/b ...(1)
 where a and b are integers. Squaring on
both sides of equation (1),
 2 = a2/b2
 i.e. 2b2 = a2
2. Proofs by Contradiction .

 This shows that L.H.S. is even (i.e.

multiple of two). Hence R.H.S. is also
even. Now if we write a = 2 k then,
 2b2 = (2 k)2 = 4k2
 b2 = 2 k 2
 This means, even b is an even number.
This is contradiction our assumption that
a/b simplified to lowest term because a
and b are both even. So √2 cannot be
rational.
3. Proof by Counter Example

 In order to prove certain statements, we

need to see all possible conditions in which
that statement remains true. There are some
situations in which the statement can not be
true.
 For example -
 There is no such pair of integers such that
 a mod b = b mod a
 Proof:
 Consider a = 2 and b = 3. Then clearly
 2 mod 3≠ 3 mod 2
Inductive proof
 Inductive proofs are special proofs based
on some observations. It is used to prove
recursively defined objects.
 This type of proof is also called as proof by
mathematical induction.
 Steps:
 1. Basis: In this step we assume the
lowest possible value.
 2. Induction hypothesis: In this step we
assign value of n to some other value k.
Inductive proof
 3. Inductive step: In this step, if n = k
is true then we check whether the result
is true for n = k + 1 or not.
Inductive proof or mathematical
Induction

 To prove by induction that

 Proof :
 Basis Step : n=0;

 Induction hypothesis : Assume n=k is

true
Inductive proof or mathematical
Induction

 Inductive step: Now, we need to show

that the formula is true for k+1. That is,
we need to prove:

 Start with the left-hand side of the

equation:
Inductive proof or mathematical
Induction

 Using the inductive hypothesis, we have:

 Combine the terms over a common

denominator:

 Factor out (k+1) from the numerator:

Inductive proof or mathematical
Induction

 Prove that
Using method of induction for n>0
 Proof:
 Basis Step : Let n=1;
 LHS = 1,
 RHS =

 LHS = RHS
Inductive proof or
mathematical Induction*********

 Prove that 12 + 22 + 32 + ... + n2 =

Σni=1 i2
 =

 using mathematical induction.

 Solution :

 Let n = 1

 L.H.S. = 12 = 1
Inductive proof or
mathematical Induction
*********

 R.H.S. = 1(1+1)(2.1+1) /6 = 6/6

 R.H.S. = 1

 As L.H.S. = R.H.S it is proved for n =

1.
 2. Induction hypothesis - Let, n =

k.
 Then

 12 + 22 + …….. + k2 = Σk i 2
=
is true
i=1
Inductive proof or
mathematical Induction*********

 3. Inductive step :
 Now, we will try to prove it for n = k +

1. Hence we will substitute n by k + 1.

 12 + 22 + 32 +............+ (k+1)2 =

Σk+1i=1 i2 =

 L.H.S. = 12 + 22 + 32 +...... + k2 +
(k+1)2
Inductive proof or
mathematical Induction
*********

 3. Inductive step : Now, we will try

to prove it for n = k + 1. Hence we
will substitute n by k + 1.
 12 + 22 + 32 +............+ (k+1)2 =

Σk+1i=1 i2 =
Inductive proof or
mathematical Induction
*********

 Starting with the left-hand side:

Inductive proof or
mathematical Induction
*********

 Factor out (k+1):

Inductive proof or
mathematical Induction
*********

 Factor the quadratic expression:

Inductive proof or
mathematical Induction
*********

 Prove that for every integer n≥0 the

number 42n+1 + 3n+2 is multiple of 13.
 Solution: We will prove this using
method of induction.
 Basis - Assume n = 1
 P(1) = 42(1)+1+31+2
 = 43 + 33
 = 91 which is = 13×7
 Hence 42n+1 + 3n+2 is multiple of 13 when
n = 1.
Inductive proof or
mathematical Induction
*********

 Inductive step - Assume n = k

 P(k) = 42k+1 + 3k+2

 P(k) = 13 m = 42k+1 + 3k+2

 where m is some integer.

 Thus we assume that given

expression is true for P(k).

 Now we will prove that is also true

for P(k+1).
Inductive proof or
mathematical Induction
*********

 Let, n = k +1 then
 P(k+1) = 42(k+1)+1 + 3(k+1)+2

 = 42k+3 + 3(k+3) 42k+1 + 3k+2

 = 42k+1+2 + 3(k+2+1)

 = 42k+1 42+ 3(k+2) 31 42k+1 + 3k+2 = 13

 16 . 42k+1 + 3 . 3(k+2) m
4 2k+1
 16 . (13 m - 3k+2)+ 3 . 3(k+2)
= 13 m - 3 k+2

 16 . 13 m – 16 . 3k+2+ 3 . 3(k+2)
Inductive proof or
mathematical Induction
*********

 16 . 13 m – 16 . 3k+2+ 3 . 3(k+2)
 16 . 13 m – 13 . 3k+2

 Notice that both terms are multiples

of 13:
 13 (16m - 3k+2)
Inductive proof or
mathematical Induction*********

 Prove for every n ≥ 1 by mathematical

induction Σn i3 = {n(n+1)/2}2.
 Solution:
 Basis step
 Assume
 n = 1 then
 L.H.S. = (1)3 = 1
 R.H.S. = [1(1+1) /2]2 = (2/2)2 = 1
 As L.H.S. = R.H.S.
 Thus the given expression is true for n = 1.
Inductive proof or
mathematical Induction *********

 Inductive Hypothesis
 Assume n = k is true
 Then 13+23+33 +...+ k3 = [k(k+1) /2]2 is
also true, for some k in the set of positive
integers.
 13+23+33 +...+k3 = [k(k+1) /2]2 …..(1)
 We will show that if n = k is true then
it implies n = k + 1 is also true.
 L.H.S. = 13+23+33+ ... + k3+(k+1)3

 R.H.S. = [(k+1)(k+2) /2]2

Inductive proof or
mathematical Induction
*********

 From equation (1)

 [k(k+1) /2]2 +(k+1)3 = k2(k+1)2 /4 +(k+1)3
 = k2(k+1)2 +4(k+1)3 /4
 = k2(k2+2k+1)+4(k3 +3k2+3k+1)
 = k2(k2 +2k+1)+4k3+12k2 +12k+4 /4
 = k4+2k3+k2 +4k3 +12k2 +12k+4
 L.H.S. = k4+6k3 +13k2 +12k+4 /4 …..(2)
Inductive proof or
mathematical Induction
*********

 Now consider R.H.S.

 R.H.S. = [(k+1)(k+2) /2]2
 = (k+1)2 (k+2)2 /4
 = (k2 +2k+1)(k2 +4k+4) /4
 = k4+4k3 +4k2 + 2k3 +8k2 +8k+k2 +4k+4 /4
 R.H.S. = k4 +6k3 +13k2 +12k+4 /4 …..(3)
 From equation (2) and equation (3) we
get
 L.H.S. = R.H.S. is proved for n = k +1

 Thus given expression is true.

Finite State Machine
*********

 Symbol – a,b,c,0,1,2,3
 Alphabet - Σ (sigma) - Collection of symbols

 Eg – Σ = {a,b} , {0,1}
 String – Finite Sequence of symbols chosen from

some alphabet.
 Eg – 01010100 over the Σ = {0,1}
 The Empty String - the string with zero occurrences of

symbols . Denoted by - ε
Finite State Machine
*********

 Length of a String - the number of positions for

symbols in the string
 Eg – 00101 has length 5
 Notation for the length of a string w is |w|.
 Powers of an Alphabet
 We define Σk to be the set of strings of
length k each of whose symbols is in Σ.
 Σ0 = {ε}
 Σ = {a,b} = Σ1 = {a,b}
Finite State Machine
*********

 Powers of an Alphabet
 Σ = {a,b} = Σ2 = {aa,ab,ba,bb}
 Σ = {a,b} = Σ3 = {aaa, aab, aba, abb bbb,

bba, baa, bab}

 Σ = {a,b} = Σ* = Σ0 U Σ1U Σ2 U Σ3 U

…..
 Σ* = {ε,a,b, aa,ab,ba,bb, aaa, aab, aba, abb bbb,

bba, baa, bab,………..}

Finite State Machine
*********

 Powers of an Alphabet
 Σ = {a,b} = Σ+ = Σ1U Σ2 U Σ3 U …..
 Σ+ = {a,b, aa,ab,ba,bb, aaa, aab, aba, abb bbb,
bba, baa, bab,………..}
 Concatenation of Strings
 Let x and y be strings Then xy denotes the concatenation
of x and y
 Eg – x = 10010, y= 110
 xy = 10010110 yx = 11010010
Finite State Machine
*********

 Language
 A set of strings all of which are chosen

from some Σ* where Σ is a particular

alphabet is called a language.
 Example : Σ = {0,1}

 L1 = set of all string of length 2

 = {00,01,10,11}
 L2 = set of all string of length 3

 = {000,001,010,011,100,101,110,111}
Finite State Machine
*********

 L3 = set of all string that begin with

0
 = {0,00,01,000,010,011,0000…..}
 Language L1 and L2 Finite

 L3 infinite

 Ø the empty language is a language

over any alphabet.

 Ø ≠ {ε}
Finite State Machine
*********

 Operation on languages:
 1) Product or Concatenation

 L1 = {an b , n≥0}
 L2 = {0n 1 , n≥0}
 L1L2 = {anb0n1}
 2) Reversal (LR)

 The Reversal of a language is a set of all reverse

strings
 L = {abc}

 LR = {cba}
Finite State Machine
*********

 Grammar :
 A grammar of a language (G) is

defined as,
 G(V,T,S,P)
 V = Set of Non terminals

 T = Set of Terminals

 P = set of production rule

 S = Starting symbol
Finite State Machine
*********

 Grammar :
 A grammar of a language (G) is

defined as,
 G(V,T,S,P)
 V = Set of Non terminals

 T = Set of Terminals

 P = set of production rule

 S = Starting symbol
Finite State Machine
*********

 Eg :
 S → aAb / a

A → ε

 V = {S,A}

 T = {a,b, ε}

 P = S → aAb / a

A →ε
S = S
Finite Automata
*********

 It is a mathematical model that

accepts the set of input and
process through set of state and
generate the output.
Push
OFF ON

Push
Finite Automata
*********

 Definition
 A finite automata is a collection of 5-tuple

(Q, Σ, δ, q0, F) where,

 Q = finite set of states, which is non empty.
 Σ = input alphabet, indicates input set.

 q = initial state and q is in Q i.e. q ϵ Q.

0 0 0
 F = set of final states.
 δ = transition function or a mapping

function. Using this function the next state

can be determined.
Representation Finite
Automata *********

 Two ways
 Transition diagram
 Transition table
 Transition diagram:
 It is directed graph associated

with vertices of the graph

corresponds to the state of the
finite automata.
Representation Finite
Automata *********

Q= {q0,q1}
Σ = {0,1}
δ = q0X0 = q0
q0 = stating q0X1 = q1
state q1x0 = q0
q1 = final state q1x1 = q1
Representation Finite
Automata *********

 Transition table:
 States are represented in rows

and inputs are represented in

columns.
 Initial state represented by arrow

→
 Final state represented by *
Representation Finite
Automata *********

Stat Input
e 0 1

→q0 q0 q1
Representation Finite
Automata *********

 Language acceptance by
Finite automata
 A string x is accepted by the

automata, M= (Q, Σ, δ, q0, F)

only if δ(q0,x)=P for some in x
the language accepted by M
which is denoted by L(M)
Representation Finite
Automata *********

 Language acceptance by
Finite automata
 A string x is accepted by the

automata, M= (Q, Σ, δ, q0, F)

only if δ(q0,x)=P for some in x
the language accepted by M
which is denoted by L(M)
Representation Finite
Automata *********

Check whether the i/p string 110101 is

accepted by finite automata
 Q=
 Σ=
 q0 =

 F =
Stat Input
e 0 1

→q0* q2 q1
q1 q3 q0
q2 q0 q3
 1 1 0 1 0 1

 q1 q2 q0 q2 q3 q1 q0

 q0 final state so this string accepted by

finite automata
 Try 1101