Module 2 – Recursive

Algorithms & Method of Solving

Recurrences
Jayavignesh T
Asst Professor
SENSE
 Divide-and-Conquer

• The most-well known algorithm design strategy:

1. Divide instance of problem into two or more smaller
instances

2. Solve smaller instances recursively

(Base case: If the sub-problem sizes are small enough, solve
the sub-problem in a straight forward or direct manner).

3. Obtain solution to original (larger) instance by combining

these solutions
• Type of recurrence relation
Divide-and-Conquer
Divide-and-Conquer Technique (cont.)
 Recursive Algorithm
• Any problem can be solved either by writing recursive
algorithm or by writing non-recursive algorithm.

• A recursive algorithm is one which makes a recursive call to

itself with smaller inputs.

• We often use a recurrence relation to describe the running

time of a recursive algorithm.

• Recurrence relations often arise in calculating the time and

space complexity of algorithms.
 Recurrence Relations contd..
• A recurrence relation is an equation or inequality that
describes a function in terms of its value on smaller inputs or
as a function of preceding (or lower) terms.

1. Base step:
– 1 or more constant values to terminate recurrence.
– Initial conditions or base conditions.
2. Recursive steps:
– To find new terms from the existing (preceding) terms.
– The recurrence compute next sequence from the k
preceding values .
– Recurrence relation (or recursive formula).
– This formula refers to itself, and the argument of the
formula must be on smaller values (close to the base value).
 Recurrence Formula :
Ex 1 Fibonacci Sequence
• Recurrence has one or more initial conditions and a
recursive formula, known as recurrence relation.
• Fibonacci sequence f0,f1,f2…. can be defined by the
recurrence relation

• (Base Step)
– The given recurrence says that if n=0 then f0=1 and if n=1
then f1=1.
– These two conditions (or values) where recursion does not
call itself is called a initial conditions (or Base conditions).
 Ex : Fibonacci Sequence contd..
• (Recursive step): This step is used to find new
terms f2,f3….from the existing (preceding)
terms, by using the formula

• This formula says that “by adding two previous

sequence (or term) we can get the next term”.
Ex 2 : Factorial Computation
 Recurrence Relation for
Factorial Computation

• M(n)
– denoted the number of multiplication required to
execute the n!
• Initial condition
– M(1) = 0 ; BASE Step
• n>1
– Performs 1 Multiplication + FACT recursively called
with input n-1
Recurrence Relation for
Factorial Computation
Example 3
 Example 3

• Let T(n) denotes recurrence relation - number

of times the statement x=x+1 is executed in
the algorithm

x+1 to be executed T(n/2) additional times

Example 3

Performs TWO recursive

calls each with the
parameter at line 4, and
some constant number of
basic operations
 Recurrences and Running Time
• An equation or inequality that describes a function in terms of
its value on smaller inputs.
T(n) = T(n-1) + n
• Recurrences arise when an algorithm contains recursive calls to
itself

• What is the actual running time of the algorithm?

• Need to solve the recurrence
– Find an explicit formula of the expression
– Bound the recurrence by an expression that involves n

15
 Recurrent Algorithms - BINARY-SEARCH
• for an ordered array A, finds if x is in the array A[lo…hi]

Alg.: BINARY-SEARCH (A, lo, hi, x)

1 2 3 4 5 6 7 8
if (lo > hi) 2 3 5 7 9 10 11 12
return FALSE
mid  (lo+hi)/2 mid
lo hi
if x == A[mid]
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x)

16
 Example
• A[8] = {1, 2, 3, 4, 5, 7, 9, 11}
– lo = 1 hi = 8 x = 7
1 2 3 4 5 6 7 8

1 2 3 4 5 7 9 11 mid = 4, lo = 5, hi = 8

5 6 7 8

1 2 3 4 5 7 9 11 mid = 6, A[mid] = x Found!

17
 Another Example
• A[8] = {1, 2, 3, 4, 5, 7, 9, 11}
– lo = 1 hi = 8 x=6
1 2 3 4 5 6 7 8

1 2 3 4 5 7 9 11 mid = 4, lo = 5, hi = 8
low high

1 2 3 4 5 7 9 11 mid = 6, A[6] = 7, lo = 5, hi = 5
low high
1 2 3 4 5 7 9 11 mid = 5, A[5] = 5, lo = 6, hi = 5
NOT FOUND!

1 2 3 4 5 7 9 11
high low

18
 Analysis of BINARY-SEARCH
Alg.: BINARY-SEARCH (A, lo, hi, x)
if (lo > hi) constant time: c1
return FALSE
mid  (lo+hi)/2 constant time: c2
if x = A[mid]
constant time: c3
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x) same problem of size n/2
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x) same problem of size n/2

• T(n) = c +T(n/2)
– T(n) – running time for an array of size n
19
 Methods for Solving Recurrences
• Iteration method
– (unrolling and summing)
• Substitution method
• Recursion tree method
• Master method

20
 Method of Solving Recurrences
• Iteration Method :
– Converts the recurrence into a summation and then relies on
techniques for bounding summations to solve the recurrence.
• Substitution Method :
– Guess a asymptotic bound and then use mathematical
induction to prove our guess correct.
• Recursive Tree Method :
– Graphical depiction of the entire set of recursive invocations
to obtain guess and verify by substitution method.
• Master Method :
– Cookbook method for determining asymptotic solutions to
recurrences of a specific form.
 The Iteration Method
• Convert the recurrence into a summation and
try to bound it using known series
– Iterate the recurrence until the initial condition is
reached.
– Use back-substitution to express the recurrence in
terms of n and the initial (boundary) condition.

22
Iteration Method - Example 1
Iteration Method - Example 1
Binary Search – Running Time
T(n) = c + T(n/2)
 Binary Search – Running Time
T(n) = c + T(n/2)
T(n) = c + T(n/2)
= c + c + T(n/4)
= c + c + c + T(n/8)
Assume n = 2k
T(n) = c + c + … + c + T(1) = c log2 n
+ T(1) k times
= O(log n)
Recap : Arithmetic Series
Recap : Geometric Series
Problem 3
Iteration method
Iteration method
 Example Recurrences

• T(n) = T(n-1) + n
– Recursive algorithm that loops through the input to eliminate one item

• T(n) = T(n/2) + c
– Recursive algorithm that halves the input in one step

• T(n) = T(n/2) + n
– Recursive algorithm that halves the input but must examine every item in
the input

• T(n) = 2T(n/2) + 1
– Recursive algorithm that splits the input into 2 halves and does a constant
amount of other work

• T(n) = T(n/3) + T(2n/3) + n

Logarithmic Formulae - Revisit
General form of D & C Strategy
 Divide and Conquer – Recurrence form

T(n) – running time of problem of size n.

If the problem size is small enough (say, n ≤ c for some constant c), we have a base case.

The brute-force (or direct) solution takes constant time: Θ(1)

Divide into “a” sub-problems, each 1/b of the size of the original problem of size n.

Each sub-problem of size n/b takes time T(n/b) to solve

Total time to solve “a” sub-problems = spend aT(n/b)

D(n) is the cost(or time) of dividing the problem of size n.

C(n) is the cost (or time) to combine the sub-solutions.

 Iteration Method
Unroll (or substitute) the given recurrence back to itself until a
regular pattern is obtained (or series).

Steps to solve any recurrence:

1. Expand the recurrence

2. Express the expansion as a summation by plugging the recurrence

back into itself until you see a pattern.

3. Evaluate the summation by using the arithmetic or geometric

summation formulae
 Master’s method
• “Cookbook” for solving recurrences of the form:
 n
T (n) aT    f (n)
b
where, a ≥ 1, b > 1, and f(n) > 0

Idea: compare f(n) with nlogba

• f(n) is asymptotically smaller or larger than nlogba by a

polynomial factor n

• f(n) is asymptotically equal with nlogba 39

 Why n log a
b ?  n
T (n) aT    f (n)
b

 n • Case 1:
T (n) aT  
b – If f(n) is dominated by nlogba:
 n
a 2T  2  • T(n) = (nlogba)
b 
 n
a 3T  3  • Case 3:
b 
– If f(n) dominates nlogba:
n
T (n) a iT  i  i • T(n) = (f(n))
b 

• Case 2:
• Assume n = bk  k = logbn – If f(n) = (nlogba):
• At the end of iteration i = k: • T(n) = (nlogba logn)

 bi 
T (n) a logb n
  
T  i  a logb nT (1)  a logb n  n logb a 
b 
40
 Master’s method
• “Cookbook” for solving recurrences of the form:

 n
T (n) aT    f (n)
b
where, a ≥ 1, b > 1, and f(n) > 0

Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba)

Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)

Case 3: if f(n) = (nlogba +) for some  > 0, and if

af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then:
T(n) = (f(n))
regularity condition
41
 Examples

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare nlog22 with f(n) = n

 f(n) = (n)  Case 2

 T(n) = (nlgn)
42
 Examples
T(n) = 2T(n/2) + n2
a = 2, b = 2, log22 = 1
Compare n with f(n) = n2
 f(n) = (n1+) Case 3  verify regularity cond.

a f(n/b) ≤ c f(n)
 2 n2/4 ≤ c n2  c = ½ is a solution (c<1)
 T(n) = (n2)
43
 Examples (cont.)

T(n) = 2T(n/2)n+

a = 2, b = 2, log22 = 1

Compare n with f(n) = n1/2

 f(n) = O(n1-) Case 1

 T(n) = (n)
44
 Examples
T(n) = 2T(n/2) + nlgn

a = 2, b = 2, log22 = 1

• Compare n with f(n) = nlgn

– seems like case 3 should apply

• f(n) must be polynomially larger by a factor of n

• In this case it is only larger by a factor of lgn

45
 Examples
T(n) = 3T(n/4) + nlgn

a = 3, b = 4, log43 = 0.793

Compare n0.793 with f(n) = nlgn

f(n) = (nlog43+) Case 3

Check regularity condition:

3(n/4)lg(n/4) ≤ (3/4)nlgn = c f(n),

c=3/4
46