Lect Recurrences
Lect Recurrences
Mathematical Recurrences
A recurrence is a function T(n), that is defined in terms of its values
with smaller inputs.
Example: 11
n ⎪
⎧ ⎪
Tn =
⎨⎪ >
() n ⎛
2 ⎟+
= T ⎞
⎪ ⎜ ⎠ 1
⎩ ⎝ 2 nn
= ⎝ ()
⎜ 2 ⎠ 3
Mathematical Recurrences
Recurrences are used to represent the runtime of a recursive function
Cost of a recursive algorithm is equal to the cost of non-recursive part
plus the recursive call on smaller input size.
= ⎝
⎜ b⎠
• a is the number of times a function calls itself
• b is the factor by which the input size is reduced
• f(n) is the runtime of each function usually expressed in terms of Θ. 4
Recurrence Example
Factorial (n)
{
if (n == 1)
return 1;
else
return (n * Factorial(n-1));
}
The expression:
11
n Tn
⎧ =
⎪
() ⎪
⎩()
is a recurrence. cT n n 5
= −+>
⎨ 111
Y
Recurrence Example: n
Power (n, y)
{
if (y == 0)
return 1;
else
return (n * Power(n, y-1));
}
The expression:
cn ⎪ =
⎧ Tn 0
()
= ⎩()
⎨ cT n n
⎪ −+> 6
111
Recurrence Examples
00
s n ( )= ⎧ 0 = n sn
n 0 ⎧ =
⎨ ( 1) > ⎨
csn () 0 nsnn
+− n = +−>
⎩
⎧
( 1) 0
⎧ = 1
cn ⎩
⎪ Tn
Tn =
⎪ 1
cn
⎪ ⎪
() ⎨⎪ n ⎛ > = ⎞
= 2 T ⎟+() ⎨⎪ ⎛ n
⎞
⎪ ⎝ ⎠ aT ⎟ + >⎠
⎪
⎩ 1 cn ⎜ b cn n 1
⎩
⎜ 2 ⎝ 7
Linear Search
LinearSearch ( A, key, low, high)
{
if ( A[low]=key)
return low;
else if ( low> high)
return (–1)
else LinearSearch(A,key,low+1, high);
}
9
T(n) = T(n-1) + 1
Binary Search
BinarySearch(A,low,high,key
) if (low >high)
return (–1)
mid 🡨 (low + high)/2
if (key=A[mid])
return mid
else if ( key>A[mid])
BinarySearch(A,mid+1,high,key
) else
BinarySearch(A,low,mid-1,key
) T(n) = T(n/2) + 1,
10
T(1) = 0.
Merge Sort
Statement Effort
MergeSort(A, p, r) { T(n) if (p < r) { Θ(1) q
= floor((p + r) / 2); Θ(1)
MergeSort(A, p, q); T(n/2) MergeSort(A,
q+1, r); T(n/2) Merge(A, p, q, r); Θ(n) }
}
Θ=
(1) 1
n
⎧
⎪
⎪
Tn
() ⎨⎪ ⎛ Θ >( ) 1
n
= 2 ⎟+
T ⎞
⎪ ⎜ 2 nn
⎩ ⎝ ⎠ 11
Recurrence Equation:
Θ≤
()1
cn
⎧
⎪
Tn
()
=
⎨
⎪
TnTnn
( 1) ( 2) 1
−+−>
⎩
12
Fibonacci Sequence
algorithms?
What is the F(n)
drawback in
recursive
F(n-1) F(n-2)
F(n-2) F(n-3) F(n-3) F(n-4)
F(1) F(0) 13
F(0) F(1)
The Towers of Hanoi Puzzle
Some problems are computationally too difficult to solve in
reasonable period of time. While an effective algorithm exists, we
don’t have the time to wait for its completion.
Initially:
• Three posts (named Left, Middle, Right)
• N disks placed in order on the leftmost post
Goal:
• Move all the disks on the leftmost post to the rightmost post •
The original ordering must be preserved
Constraints:
14
• Only one disk may be moved at a time.
• Disks may not be set aside but must always be placed on a post. •
A larger disk may never be placed on top of a smaller disk.
15
Making an Algorithm
16
17
11
n Tn
⎧ =
⎪
18
() Tnn
= ⎩()
⎨ 2111
⎪ −+>
Solving Recurrences
Three methods are used in “solving” recurrences
• The Substitution Method
• The Recursion Tree Method
• The Master Method
20
Substitution Method
• Substitution method has two steps
• Guess the form of the solution
• Use induction to prove that the solution is correct
Substitution Method •
Quick Sort and Merge Sort Evaluation
n
T n = T )+
( ) 2. n
( 2
nn
n
T=+
( 2 2 ) 2. ( T
2
)
2
n nn
T=+
( (T
) 2.
)
232
2
2 2
nn
n
T=+
( T ...
) 2. (
)
343
2
2 2
nn
n 22
T k ) 2. (
2(− − = + k k
T )
11
2 2
Substitution Method
n
n 2
T n = T +Θ = )+ +
( ) 2. ( 2 nT
)()2.( 2
2
nn
n
2
T n = T )+ +
2
()2 .(
2 nn
n
3
Tn=T)+++
()2. 3
nnn
( 2
)2.(
k
T n T ( ... n
= )+ + +...+ n n n
−
2 24
k
43 k
1 44 times
k n
TnTk
( ) = 2 . ( + nnk) .
2 k that : = 2 ⇒log2 =
Let us suppose
kn
Hence,T(n) = n.T(1)+n.log2 ( ) ( .log ) T n = Θ n 2
n
n = n+n.log2
n
23
Substitution Method
• Quick Sort Worst case: pivot is smallest/largest element all the time.
T(n) = T(n-1) + cn
T(n-1) = T(n-2) + c(n-1)
T(n-2) = T(n-3) + c(n-2)
…
T(n-k) = T(n-k-1) + c(n-k)
n
n nn
(2) ( 4
T=⋅+)3(
2
4 T 4 ) 4
)3(
(3) (n n
n and so on
T=⋅+
T )
232
4
4 4
25
(4) ( n nn
T 3(
= ⋅ + kkk)
T )
−−11
4
4 4
Substitution Method
n value of ( 4
T )from (2) to (1)
Nowsubstitute the
⎡ 3(Tn nn
(5) ( ) 3 T ⎤
=+
⎢ 4 ) + n
⎥⎦
⎣ 2
4
3( 3
2
n
=⋅+⋅+
T 4 )( ) nn
2 4
n
substitute T
the value of ( )from (3) to (5) have 2
4 equation, we
=⋅+) 3
4 (
⎢ nn
⎣⎡
nT++⎥
T ⎦⎤ ( ) 3 3 (3 2
2
4 4 nn 3
n
) 3 3 3
k120kk
−−
TnT
( ) = 3 ⋅ ( + + +⋅⋅⋅⋅+ +
)( )(
nnn
4 4 4
k
)( 4 4
k
= )( )
26
Let us suppose that : = 4 ⇒ log 4 n n k
n ⎡ 3 3 3 ⎤
kk
2 −1
TnT
()3()1(
=⋅+++++
n ⎢ )( ) ( ) ⎦
n ⎣ 4 4 .... 4
⎥
Substitution Method
n ⎡ 3 3 3 ⎤
kk
2 −1
TnT
()3()1(
=⋅+++++
n ⎢ .... ⎥
n ⎣ 4 4 ( 4
)( ) ) ⎦
⎡ ⎢ 3 ⎤⎥ − k
k 1(− 1
k ) 21
x
k
−
4
T n T n x x x ( ) 3 (1) 1 (
=⋅+⋅ ) 1 ... 1 (
⎢ ⎥ Θ )
3
k ⎢ ⎥ ++++ 1
1 4
k
⎣ ⎥ =⋅ −
⎢ −
3 ⎦ x
T(n) = 3 ⋅1+ 4n(1− T = ) (1) 1
k kk n 3
Θ k
43 3 k
k
3 4 (n
k =+ −
=+⋅−=+⋅−)Tnnn
( ) 3 4 (1 ) 3 4 (1 k
n n
4 )
3 4( 3 ) =1⋅3 +4n−4⋅3
kk= + n −
kk
= 4 −3⋅3 n
n
k3 log 4
= 4⋅n−3⋅3 27
T(n) = 4n−3⋅n
log
Hence T(n)∈θ(n) 4
Substitution A HardExample
Method 1 if 1
⎪
n Tn where 0 < a <
⎧ = b
() ⎨ ( ) wise
= aT n other
⎪ b n
⎩ +
n n n
T(n) = aT( ) + n = a⋅a⋅T( )+ ⋅ + 2
Solut a ba
b b
ion: 2 n a ⎡ n n ⎤
n
= a ⋅T( ) + ( ) + 2 n n
2
a aT + +
=()+() ⎥
3 n b2 ⎣
b 32b b
b
a
a ⎢ ⎦ n n
= a T( ) + ( ) + ( ) +
bk nna
n b a
3
n b a
−1 −2 1
TnaT
kk
( ) ( ) ( ) ( ) .... ( ) = + + + + +
k b ⎡ n
b b⎤
k 28
n a a n a nn
b
2 −1
k
=+++++
aT ( ) 1 ( ) ( ) .... ( )
b k n ⎢ ⎣
⎥ ⎦
b b b
Substitution Method
⎡ a a ⎤
kk
−1
TnaTn
( ) (1) 1 ( ) .... ( )
=⋅++++
k
= ⋅ + ( )( ) = +
k
1 ( )( ) ba
an
an
ba b n −
− b b bab−
b k
−
= kk ⋅+
b a⋅ ()
kkk
⋅− a
=+
b a
ba− ba−
ba ba
− −
b b log
a
a
n
a
()()⋅
k
log
Tn ⋅− ba a − ⋅− ba ⋅ ba b
− − 29
b
= b ba n n
= −
⇒T(n)∈θ(n)
Substitution A MoreHardExample
Method 1 if 1
Tn
n =
⎧
⎪
() = ⎨ 3( n 2
⋅+
⎪ T otherwis
⎩ 4
) e cn n
Solution 2
:
(1) ( ) 3 ( cn T n = ⋅T
+
(2) ( 4n
n ) n
2
T=⋅+)3(
( T 2 c
4 4 n
)(
(3)
n 4 n ) 2
T = ⋅ +) 3 (
T )(c)
232
4
4n n and so
(4) ( n 4 2 on
T=⋅+)3(
T )(c)
343
4
4 4 30
(5) ( n n n 2
T
= ⋅ + kkk) 3 (
T )(c )
−−11
4
4 4
Substitution Method
n value of ( 4
T )from (2) to (1)
Nowsubstitute the
⎡ n n ⎤ 222
n 22
n
++
(6) ( ) 3 3 ( cn T n T + = ⎥
=+
)(
⎢ c ) 3 cn ⎦ 2
(c
⎣ 4 ( T )3 4
2
4 4 )
n
substitute T
the value of (6), we have
( 4 2
)from (3) to
2
n n ⎤ n
⎡
222
TnT+⋅+
⎥ ( ) 3 3 ( cn = +
)(
⎢ c )3( ⎦ )
4
⎣ 324 c
4
n 2 n n
3 222
= T + + ⋅ + 3 ( cn ) 3 (
4 c 4 2 )3(c )
3 4
n 3 22 3 122
3
= T + + + 3 ( cn cn cn
4 )( 32 4 )( ) 4
2
31
k k − −1
k n ⎡ 3 3 3 ⎤
TnT 2 02
( ) 3 ( cn = + + +⋅⋅⋅⋅+
4 )(⎢ )( 2 4 ) 2
⎦
k ⎣ 4
)( 2
4 ⎥
Substitution Method
k
Let ussuppose that n can be expressed as n = 4
3 33 ⎤
⎡
kk
2 2 −1
T n T cn
( ) 3 (1) 1
= + + + +⋅⋅⋅⋅+
k
3 ( )( ⎥
1(
2
16 16
⎡ ⎢
⎢ ⎤ 16 ⎦ )
3
16 −
⎣ 16 ) ⎥
k
k 3
k 2
T n cn ) = + cn ⋅ −
()31(
(1
=⋅+ ⎢ 1 − ⎥
⎣ ⎥ 13 16 2
⎢ )
⎢ ⎥ ⎦
3 16 k k
22
⇒ = 16 n
k
since 4 = k n ⇒ = (4 ) n
22
k
⇒ = (4 ) n
2
16 16 2 16
k
3 (1
16
32
n 3 )=⋅+−
− k
k2k
c n c)3
=+⋅(3)
k
13 n − 13
cn n 13 13
2 =+⋅c⋅
(
16 n 16 2 log
)
()=+− 16 (1
32
16 3
(1
2 log
)3 = cn + − c n
4
T n cn c 4
13 13
13 13
2
T n ∈θ n
Hence ( ) ( )
Recursion Tree Method
• Although substitution method can provide a sufficient
proof that a solution to a recurrence is correct, sometimes
difficult to give a good guess.
c.n2
T(n/4) T(n/4) T(n/4) c.n2
2
c.(n/4)
2
c.(n/4)
36
T(n/16) T(n/16) T(n/16)
T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16)
2
2 c.(n/4)
c.(n/4)
(3/16).c.n2
2 2 2
c(n/16) c(n/16)
2
c(n/16) 2
c(n/16) 2
c(n/16) (3/16) .c.n2
(3/16)2.c.n2
(3/42)k-1 cn2
k
T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(n/4 )
T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) 38
Θ(nlog43)
⎤
3
⎡ 3 3 3
log
)
T n n cn k
0
1 ()()(
4 12
−
= Θ + + +⋅⋅⋅⋅+
⎢ )( )( ⎥ ⎦
⎣ 2 4 4 4
2 2
1
16
)()
log 3 2 log 3 2 4 4
T n n cn = Θ n + cn ( ) ( ) (
≤Θ+ 1(
3 13
39
− ) Hence T n =
16 Οn
2
:()()
Recursion Tree Method
T(n) = 2T(n/2) + n2.
40
T(n)
44
n2
T(n/4) T(n/2)
45
n2
(n/4)2 (n/2)2
T(n/16) T(n/8) T(n/8) T(n/4)
46
n2
(n/4)2 (n/2)2
(n/16)2(n/8)2(n/8)2(n/4)2
Θ(1)
47
n 2
n
2
(n/4)2 (n/2)2
(n/16)2(n/8)2(n/8)2(n/4)2
Θ(1)
48
n 2
n
2
(n/4)2 (n/2)2
8)2(n/4)2 Θ(1) 2
(n/16)2(n/8)2(n/
5
16 49
n
Recursion Tree
Method
50
Recursion Tree
Method
2
n
2 (n/2)2 5 n
(n/4)
16 2
/4)2 2
(n/16)2(n/8)2(n/8)2(n 25
n 256
…
Θ(1)
(1 ( ) ( ) )
2 2 3
5 5 5
Total =
n + + + +Λ
16 16 16 51
= Θ(n2) series
geometric
Master Method
52
Example 1
T(n) = 9T(n/3) + n
• f(n) = n,
• nlogba= n2
• compare f(n) = n with the cost of recursiveness..
n = O(n2 -ε) (ε =1, so f(n) =nlogba),
53
54
Example 3
55
Example 4
• T(n) = T(n/2) + n2
0
• nlogb a = nlog2 1 = n = 1 where ε=2
• Since f(n) = O(nlog2 1 + 2)= n2, where ε=2, case III applies: • Case
III if f(n)= Ω(nlogB a + ε) for some constant ε>0, and if
af(n/b)<=cf(n) for some constant c<1 and all sufficient large
n, then T(n) = Θ (f(n))
• af(n/b)<=cf(n)
• n2/4 <= cn2 🡪 1/4<=c
56
• Thus the solution is T(n) = Θ(n2)
Example 5
T(n) = 4T(n/2) + n2
D(n) = n2, nlogba= nlg4 = n2
compare both sides, D(n) with n2
polynomially equal, case 2 holds, T(n) = θ(n2lgn)
57
Example 6
T(n) = 7T(n/3) + n2
• f(n) = n2, nlogba= nlog37 = n1+ ε
• compare f(n) = n2 with n1+ ε, n2 = Ω(n1+ε) so f(n) is
polynomially larger
• Since f(n) is a polynomial in n, case 3 holds and T(n) = θ(n2) 58
Example 7
T(n) = 7T(n/2) + n2
T(n)=2T(n/2) + n
A=2 b=2 f(n)=n
60
Example 9
• T(n) = 4T(n/2) + n3
• a=4, b=2, f(n) = n3
2
• nlogb a = nlog2 4 = O(n ) where ε = 1
• Since f(n) = O(nlog2 4 + ε), where ε=1, case III applies: • Case III
if f(n)= Ω(nlogB a + ε) for some constant ε>0, and if af(n/b)<=cf(n)
for some constant c<1 and all sufficient large n, then T(n) =
Θ (f(n))
• a f(n/b)= 4(n/2)3 <=4/2n3=c f(n) for c=4/2
• af(n/b)=4/8n3<=4/2n3=c f(n)
• af(n/b)=n3/2<=2n3=c f(n)
f(n)= n3
• Thus the solution is T(n) = Θ(n3)
61
4f(n/2) = 4(n/2)3
af(n/b)=a (n/b)3
Practice Questions
• T (n) = 3T (n/2)+ n2
• T (n) = 4T (n/2)+ n
• T (n) = T (n/2) + 2n
• T (n) = T (n/2) + 1 Binary Search
• T (n) = 2nT (n/2) + n
• T (n) = 16T (n/4)+ n
• T (n) = 2T (n/2)+ n log n
• T (n) = 2T (n/4)+ n0.51
• T (n) = 0.5T (n/2)+ 1/n
• T (n) = p2T (n/2) + log n
• T (n) = 3T (n/2)+ n
• T (n) = 3T (n/3)+ pn
• T (n) = 4T (n/2)+ cn
• T (n) = 3T (n/4)+ n log n
• T (n) = 3T (n/3)+ n/2
• T (n) = 6T (n/3)+ n2log n
• T (n) = 4T (n/2)+ n/ log n
• T (n) = 64T (n/8)− n2log n
• T (n) = 7T (n/3)+ n2
62
• T (n) = 4T (n/2)+ log n
• T(n)=2T(n-1)+O(n)
• T(n) =3T(n/3) + O(n)