Design and Analysis of
Algorithm
Recurrences
A recurrence is an equation or relation that defines a
function in terms of lower order arguments, with the
following properties
(i) The
function is defined over a set of natural numbers
{0,1,2, 3,……}
(ii)The
definition includes a base value, called boundary
condition or initial condition.
Example(1): The factorial function f(n)= n! =1.2.3….(n-1)n
canbe expressed as a recurrence:
◦ f(n) = n . f(n – 1)for n>1
◦ f(0) =1(initial condition)
Methods for Solving Recurrences
Recursion Tree
Iteration Method and Substitution
Method
Master Theorem
Example
void Print(int n)
{
if(n>0) 1
{
cout<<n; 1
Print(n-1); T(n-1)
}
} T(n) = T(n-1) + 2
or
T(n) = T(n-1) + 1
Suppose n=5 Print(5)
Recursion Tree
Print(4) 5
void Print(int n) 4
Print(3)
{
if(n>0) Print(2) 3
{
cout<<n; Print(1) 2
Print(n-1);
Print(0) 1
}
} X
Print “n” times.
T(n) =O(n) Funtion call: n+1 times
Example: The solution to the recurrence
1 when n =0
T(n) = T(n-1) + 1when n > 0
Solve it also by using iteration or
substitution method.
Substitution Method
TheSubstitution Method for solving the Divide-and-
Conquer recurrence consists of the following steps
Step #1: In the recurrence, plug in progressively the values
n/b, n /b2, n /b3…, on the right-hand side of the equation.
Step #2: Repeat the procedure until the base case is reached
Step #3: The iterative steps would generate some kind of
pattern or a series. Perform the summation to express the
running time in closed form.
Step #4: Analyze the summation to express the running time
in asymptotic notation
1.Substitution Method
T(n) = T(n-1) + 1 T(n-1)= T(n-1-1)+1
T(n-1)=T(n-2)+1
T(n) = [T(n-2) + 1] + 1
T(n) = T(n-2) + 2
T(n-2)= T(n-2-1)+1
T(n-1)=T(n-3)+1
T(n) = [T(n-3) + 1] + 2
T(n) = T(n-3) + 3
T(n) = T(n-k) + k
1. Substitution Method
T(n) = T(n-k) + k
T(n) = T(n-n) + n
T(n) = T(0) + n
T(n) = 1+ n
T(n) =O(n)
The Substitution Method
Examples
Example(1): The running time of binary search algorithm is given recurrence
T(1) = c ( constant)
T(n) = T(n/2) + c, n > 1
Here T(n/2) is time to search left-half or right-half of a sorted array, c is the
combined cost of comparing one key and finding the middle element in the array.
Solution is as follows:
Initially, T(n) = T(n/2) + c = T(n/21) + c
Substituting for T(n/2), T(n) = T(n/4) + 2c
= T(n/22) + 2.c
Again substituting for T(n/4), T(n) = T(n/8) + 3c
= T(n/23) +
Continuing, after kth step, T(n) 3.c
= T(n/2k) + kk.c
The base case is reached when n / 2 =1, or n=2 . or k = lg n
k
Substituting for k , T(n) = T(1)+ lg n. c
= c + lg n. c (Closed form)
Ignoring constants, T(n) = θ( lg n) ( Asymptotic notation)
The Substitution Method
Example(2): Recurrence for Divide-and-Conquer algorithm with fixed cost, which
splits the problem into two sub-problems of equal sizes, is as follows.
T(1) = c
T(n) = 2T(n/2) + c, n > 1
Initially: T(n) = 2.T(n/2) + c
= 2.T(n/21)
Substituting for T(n/2), T(n) =+22.[2.T(n/4)+c]
0 c +c
= 4.T(n/4) + 3c
= 22T(n/22)+ (20+ 21).c
Substituting for T(n/4), T(n) = 4.[2 T(n/8)+c] + 3c
=8.T(n/8) + 7.c
= 23T(n/23)
+ (20+21+22).c
Continuing, after kth substitution,
Summing the geometric series,T(n) =
2kT(n/2
T(n) = k) + (20+2k1)+2
2kT(n/2 +2+…..2
(2k - k-1
1)c ).c (cont’d)
The Substitution Method
Examples
Example(2) cont’d :
Summing the geometric series,
T(n) = 2kT(n/2k) + (2k - 1)c (cont’d)
The base case is reached when n / 2k =1, or n=2k
Substituting for 2k , T(n) = n.T(1) +( n-1).c
Using initial condition, T(n) = n.c +n.c - c
Simplifying, T(n ) =c.(2n-1) (Closed form)
Ignoring constants, T(n) = θ(n) (Asymptotic notation)
The Substitution Method
Example (3) :The running time for merge sort is given by recurrence
T(1)= c
T(n)= 2T(n/2) + cn, n>1
In this recurrence, T(n) is the running time of sorting an array of size n, which is
split into two equal subarrays each of size n/2. T(n/2) is the running time to sort
subarray of size n/2, cn is the cost of splitting and merging the two subarrays. The
solution to recurrence is as follow:
Initially,T(n) =2T(n/2) + cn
= 21T(n/2)+cn
Substituting for T(n/2), T(n)=
2[ 2T(n/4)+ cn/2] +cn
=4T(
n/4)
+
2.cn
=22T(
n/22)
+2cn
Again substituting for T(n/4),
The Substitution Method
Examples
Example(3) cont’d:
Continuing, after kth step, T(n) = 2kT(n/2k)+ kcn
The base case is reached when n/2k = 1, i.e 2k =n, or k=lg(n)
T(n) = n.T(1) + cn.lg n
Using initial condition, T(n) = cn + cn.lg n
Simplifying, T(n) =c( n +n lg n ) (Closed form )
Discarding constant c, and the lower order term n in favor of bigger term nlg n
T(n) = θ(n lgn) (Asymptotic form)
The Substitution Method
Examples
Example(4) :The recurrence for Divide-and-Conquer algorithm, which splits a
problem into three subproblems is
T(1)= c
T(n)= 3T(n/4) + cn, n>1
Initially, T(n)= 3T(n/4)+ cn
Substituting for T(n/4), T(n)= 3[3T(n/16) + cn/4] +cn
= 9T(n/16) + cn
+cn.3/4
=32T(n/42)
Again. substituting for T(n/16),+cn[(3/4)
T(n) =9[
0 +3T(n/64)+cn/16]
(3/4)1] +cn+3cn/4
=27T(n/64)+ cn+cn.3/4+ cn.916]
=33T(n/43) + cn[(3/4)0 + (3/4)1+
(3/4)2]
Continuing, after kth step, it follows
T(n)= 3kT(n/4k)+cn[(3/4)0+ (3/4)1+(3/4) 2+ ……..+(3/4) k-1] (cont’d)
The Substitution Method
Examples
Exampe(4) cont’d:
The base case, is reached when n/4k = 1.
Taking log to base 4, k = log4 n.
T(n) = c.3 log 4 n +cn[ (3/4)0+ (3/4)1+(3/4)2+………..+(3/4) log 4 n-1 ]
The geometric series has geometric ratio 3/4, which is less than 1.
Therefore,
(3/4)0 + (3/4)1+ (3/4)2+………....(3/4)log n-1= θ(1)
Using the above relations
T(n) = c. 3 log 4 n + cn.θ(1)
By using property of logarithm, 3 log 4 n = n log 4 3, and ignoring constant
term θ(1) T(n) = cn log 4 3 + cn
Since, log 4 3 < 1, the term n log 4 3 is smaller compared to n. The term c n log 4 3 is
discarded in favor of n. Therefore,
T(n) = θ(n) (Asymptotic form)
Iteration Method
The Iteration Method can be used to find solution to Decrease-and-
Conquer recurrences. The method uses the top-down approach. Broadly,
it involves following steps:
Step#1: Use the recurrence to set up the equations for the arguments n,
n-1,…3,2,1.
Step # 2: On reaching the bottom level ( n=0 ), apply the boundary
condition
Step#3: Add the equations, and cancel identical terms on the left-hand
and the right- hand sides
Step#4: Perform summation to obtain solution in closed form or in
asymptotic notation
The Iteration Method
Examples
Example(1): The running time for a linear search of an array is given by the
recurrence
T(0)=0
T(n)= T(n-1) + c for n>0
In this recurrence, T(n) is time to search array of size n, T(n-1) time to search
subarray of size n-1, and c is the cost of searching one array cell. The solution is
determined as follows: Iterating the equation:.
T(n) = T(n-1 ) + c
T(n-1) = T(n-2) +c
…………………………
T(2) = T(1) +c
T(1) = T(0) +c
Adding both sides of the equations, and canceling the equal terms on the left- and
right-hand sides:
T(n) = T(0) + c+ c +………+ c ( n terms )
Summing the constant terms, T(n) = n.c (Closed form)
Ignoring constant, T(n) = θ(n) ( Asymptotic form)
The Iteration Method
Example(2): The running time of Selection Sort is given by the recurrence
T(0)=0
T(n) = T(n-1) + cn
Here T(n) is the time to sort an array of size n, T(n-1) is the time to sort subarray of
size n-1, and c.n is cost of finding a maximum key in the array and swapping it with
the last key. The solution to recurrence is obtained by iterating the equation, as
follows:
Iterating:
T(n) = T(n-1 ) +nc
T(n-1) = T(n-2) +(n-1)c
…………………………………
T(2) = T(1) + 2c
T(1) = T(0) +c
Adding both sides of the equations, and canceling equal terms:
T(n) = T(0)+ c(1 + 2+ 3 +………+n)
Evaluating the summation
T(n) = cn(n+1)/2 (Closed form)
Ignoring lower order term n compared to n2 , and the constant c
T(n) = θ (n2) (Asymptotic notation )
The Iteration Method
Examples
Example (3): Consider the Decrease-and-Conquer recurrence
T(0)=0
T(n) = T(n-1) + c lg n
Iterating:
T(n) = T(n-1 ) + c lg n
T(n-1) = T(n-2) +c lg((n-
1)
T(2) = T(1) +c lg(2)
……………………
T(1) = T(0) + c lg(1)
Adding the equations, and canceling equal terms on the left- and right-hand sides:
T(n) = T(0)+ c[ lg(1) + lg(2)+ ……+lg(n-1)+ lg(n) ]
Using initial condition and property of logarithm
T(n)= c lg(1.2…(.n-1) .n)
Using definition of factorial
T(n) =c lg( n!) (Closed form )
