Lecture three
Recurrences
Recurrences and Running
Time
An equation or inequality that describes a function
in terms of its value on smaller inputs. Such as
T(n) = T(n-1) + n
Recurrences arise when an algorithm contains
recursive calls to itself
What is the actual running time of the algorithm?
Need to solve the recurrence
Find an explicit formula of the expression
Bound the recurrence by an expression that
2 involves n
Example Recurrences
T(n) = T(n-1) + n Θ(n2)
Recursive algorithm that loops through the
input to eliminate one item
T(n) = T(n/2) + c Θ(lgn)
Recursive algorithm that halves the input in
one step
T(n) = T(n/2) + n Θ(n)
Recursive algorithm that halves the input but
must examine every item in the input
T(n) = 2T(n/2) + 1 Θ(n)
Recursive algorithm that splits the input into 2
halves and does a constant amount of other
work
3
Recurrent Algorithms
BINARY-SEARCH
for an ordered array A, finds if x is in the array
A[lo…hi]
Alg.: BINARY-SEARCH (A, lo, hi, x)
1 2 3 4 5 6 7 8
2 3 5 7 9 10 11 12
if (lo > hi)
return FALSE mid
lo hi
mid (lo+hi)/2
if x = A[mid]
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x)
4
Example
A[8] = {1, 2, 3, 4, 5, 7, 9, 11}
lo = 1 hi = 8 x=7
1 2 3 4 5 6 7 8
1 2 3 4 5 7 9 11 mid = 4, lo = 5, hi = 8
5 6 7 8
1 2 3 4 5 7 9 11 mid = 6, A[mid] = x
Found!
5
Another Example
A[8] = {1, 2, 3, 4, 5, 7, 9, 11}
– lo = 1 hi = 8 x=6
1 2 3 4 5 6 7 8
1 2 3 4 5 7 9 11 mid = 4, lo = 5, hi = 8
low high
1 2 3 4 5 7 9 11 mid = 6, A[6] = 7, lo = 5, hi = 8
low high
1 2 3 4 5 7 9 11 mid = 5, A[5] = 5, lo = 5, hi = 5
NOT FOUND!
1 2 3 4 5 7 9 11
high low
6
Analysis of BINARY-SEARCH
Alg.: BINARY-SEARCH (A, lo, hi, x)
if (lo > hi) constant time: c1
return FALSE
mid (lo+hi)/2 constant time: c2
if x = A[mid] constant time: c3
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)
same problem of size n/2
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi,same
x) problem of size n/2
T(n) = c +T(n/2)
T(n) – running time for an array of size n
7
Methods for Solving Recurrences
Iteration method
Substitution method
Recursion tree method
Master method
8
The Iteration Method
Convert the recurrence into a summation
and try to bound it using known series
Iterate the recurrence until the initial
condition is reached.
Require more algebraic knowledge.
In the iteration method we iteratively
the recurrence until we see the pattern .
9
The Iteration Method
T(n) = c + T(n/2)
T(n) = c + T(n/2) T(n/2) = c + T(n/4)
= c + c + T(n/4) T(n/4) = c + T(n/8)
= c + c + c + T(n/8)
Assume n = 2k
T(n) = c + c + … + c + T(1)
k times
= clgn + T(1)
= Θ(lgn)
10
Iteration Method – Example
T(n) = n + 2T(n/2) Assume: n =
T(n) = n + 2T(n/2) 2k
T(n/2) = n/2 + 2T(n/4)
= n + 2(n/2 + 2T(n/4))
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8)
… = in + 2iT(n/2i)
= kn + 2kT(1)
= nlgn + nT(1) = Θ(nlgn)
11
The substitution method
1- Guess a solution
2- Use induction to prove that the
solution works
12
The substitution method
n
T (n) aT f (n)
b
13
Substitution method
Guess a solution
T(n) = O(g(n))
Induction goal: apply the definition of the
asymptotic notation
(strong induction)
T(n) ≤ c g(n), for some c > 0 and n ≥ n0
Induction hypothesis: T(k) ≤ c g(k) for all k < n
Prove the induction goal
Use the induction hypothesis to find some
values of the constants c and n0 for which the
induction goal holds
14
Example 1: Binary Search
T(n) = c + T(n/2)
Guess: T(n) = O(lgn)
Induction goal: T(n) ≤ d lgn, for some d
and n ≥ n0
Induction hypothesis: T(n/2) ≤ d lg(n/2)
Proof of induction goal:
T(n) = T(n/2) + c ≤ d lg(n/2) + c
= d lgn – d + c ≤ d lgn
15 if: – d + c ≤ 0, d ≥ c
Example 2
T(n) = T(n-1) + n
Guess: T(n) = O(n2)
Induction goal: T(n) ≤ c n2, for some c and n ≥
n0
Induction hypothesis: T(n-1) ≤ c(n-1)2 for all k <
n
Proof of induction goal:
T(n) = T(n-1) + n ≤ c (n-1)2 + n
= cn2 – (2cn – c - n) ≤ cn2
if: 2cn – c – n ≥ 0 c ≥ n/(2n-1) c ≥ 1/(2 – 1/n)
16 For n ≥ 1 2 – 1/n ≥ 1 any c ≥ 1 will work
Example 3
T(n) = 2T(n/2) + n
Guess: T(n) = O(nlgn)
Induction goal: T(n) ≤ cn lgn, for some c and n
≥ n0
Induction hypothesis: T(n/2) ≤ cn/2 lg(n/2)
Proof of induction goal:
T(n) = 2T(n/2) + n ≤ 2c (n/2)lg(n/2) + n
= cn lgn – cn + n ≤ cn lgn
if: - cn + n ≤ 0 c ≥ 1
17
Changing variables
T(n) = 2T( n ) + lgn
Rename: m = lgn n = 2m
T (2m) = 2T(2m/2) + m
Rename: S(m) = T(2m)
S(m) = 2S(m/2) + m S(m) = O(mlgm)
(demonstrated before)
T(n) = T(2m) = S(m) = O(mlgm)=O(lgnlglgn)
Idea: transform the recurrence to one that
you have seen before
18
The recursion-tree method
Convert the recurrence into a tree:
Each node represents the cost incurred
at various levels of recursion
Sum up the costs of all levels
Used to “guess” a solution for the recurrence
19
Example
W(n) = 2W(n/2) + n2
20
Example
W(n) = 2W(n/2) + n2
21
Example
22
Master’s method
“used for solving recurrences of the form:
n
T (n) aT f (n)
b
where, a ≥ 1, b > 1, and f(n) > 0
Idea: compare f(n) with nlogba
f(n) is asymptotically smaller or larger than
nlogba by a polynomial factor n
f(n) is asymptotically equal with
23 nlogba
Master’s method
n
T (n) aT f (n) where, a ≥ 1, b > 1,
and f(n) > 0 b
Case 1: if f(n) = O(nlogba -) for some > 0, then:
T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all
regularity condition
sufficiently large n, then:T(n) = (f(n))
24
Examples
T(n) = 2T(n/2) + n
a = 2, b = 2, log22 = 1
Compare n with f(n) = n1/2
f(n) = O(n1-) Case 1
T(n) = (n)
25
Examples (cont.)
T(n) = 2T(n/2) + n
a = 2, b = 2, log22 = 1
Compare nlog22 with f(n) = n
f(n) = (n) Case 2
T(n) = (nlgn)
26
Examples (cont.)
T(n) = 2T(n/2) + n2
a = 2, b = 2, log22 = 1
Compare n with f(n) = n2
f(n) = (n1+) Case 3 verify regularity
cond.
a f(n/b) ≤ c f(n)
2 n2/4 ≤ c n2 c = ½ is a solution (c<1)
T(n) = (n2)
27
