CS300 - Introduction to Algorithm 20160021 Hanpil Kang March 21, 2016

Contents Homework 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

1 Problem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

2 Problem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2

3 Problem 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

4 Problem 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5

5 Problem 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6

1

Problem 1

Consider sorting n numbers stored in array A by first finding the smallest element of A and exchanging it with the element in A[1]. Then find the second smallest element of A, and exchange it with A[2]. Continue in this manner for the first n − 1 elements of A. Problem 1. Write pseudocode for this algorithm, which is known as selection sort.

1 2 3

Selection-sort(A, n): for i from 1 to n-1: do maxIndex := i

4 5 6 7

for j from i+1 to n: do if A[maxIndex] < A[j]: maxIndex := j

8 9 10 11

temp := A[j] A[j] := A[maxIndex] a[maxIndex] := temp Problem 2. What loop invariant does this algorithm maintain? Array from 1 through i-1 is sorted at the start of loop. Problem 3. Why does it need to run for only the first n − 1 elements, rather than for all n elements? Running on last elements change with value with itself and has no meaning. Problem 4. Give the best-case and worst-case running times of selection sort in Θ-notation. In every P case, all loops run. Therefore, Time complexity is (both best and 1 2 2 worst case) n−1 i=1 4 + n − i = 2 (n + 7n + 8) = Θ(n )

1

2

Problem 2

Although merge sort runs in Θ(nlgn) worst-case time and insertion sort runs in Θ(n2 ) worst-case time, the constant factors in insertion sort can make it faster in practice for small problem sizes on many machines. Thus, it makes sense to coarsen the leaves of the recursion by using insertion sort within merge sort when subproblems become sufficiently small. Consider a modification to merge sort in which n/k sublists of length k are sorted using insertion sort and then merged using the standard merging mechanism, where k is a value to be determined. Problem 1. Show that insertion sort can sort the n/k sublists, each of length k, in Θ(nk)worst-case time. Worst time complexity for sorting k numbers with insertion sort is Θ(k 2 ). There are n/k sublists, therefore total time complexity is Θ(k 2 )∗n/k is Θ(nk) Problem 2. Show how to merge the sublists in Θ(nlg(n/k)) worst-case time. Using heap with n/k elements can merge sublists. First, push to heap every minimal elements in sublist. In every iteration, pop minimal elements from heap and push next element in sublist (if exist). Every iteration needs Θ(lg(n/k)) and iteration runs n times. Therefore total time complexity is Θ(nlg(n/k)) Problem 3. Given that the modified algorithm runs in Θ(nk + nlg(n/k)) worst-case time, what is the largest value of k as a function of n for which the modified algorithm has the same running time as standard merge sort, in terms of Θ-notation? We have to optimal values, making nk term as nlgn, and making nlg(n/k) as nlgn first one gives k = Θ(lgn) and second term gives k = Θ(1). Largest value is Θ(lgn) Problem 4. How should we choose k in practice? We can choose k = 1, make algorithm as Heapsort. Or k = lgn.

2

3

Problem 3

Problem 1. Rank the following functions by order of growth; that is, find an arrangement of g1 , g2 , · · · , g20 of the functions satisfying g1 = Ω(g2 ), g2 = Ω(g3 ), · · · , g19 = Ω(g20 ). Partition your list into equivalence classes such that f (n) and g(n) are in the same class if and only if f (n) = Θ(g(n)). lg(lg ∗ n) nln n (lg n)2 2n! (lg n)! ln ln n 5n2 + 7n n5/2

n! 4n n2+sin n 8n + 12

10n + n20 nn + ln n n1/lg n 5lg n

√

n lg(n!) en nn

n

The sequence follows. g1 = 2n! ∼ 2n , g2 = nn + lnn, g3 = nn , g4 = n! ∼ n n n /e , g5 = 10n + n20 , g6 = 4n , g7 = en , g8 = (lg n)! ∼ 2(lg n)(lg lg n) , g9 = n5/2 , g10 = 5lg n = nlg 5 ∼ n2.32 , g11 = 5n2 +√7n, g12 = nln n, g13 = lg(n!) ∼ nlg n − nlg 2, g14 = 8n + 12, g15 = n, g16 = (lg n)2 , g17 = ln ln n, g18 = lg(lg ∗ n), g19 = n1/lg n = 2 ln n nn + ln n = 1 + n ) = 1, g2 = Θ(g3 ) n n n n! 2n! 2n! limn→∞ ( n = nlg n = 2 nlg n = 2∞ ) = ∞, g1 = Ω(g2 ) n 2 nn nn−1 limn→∞ ( = × n ≥ n) = ∞ g3 = Ω(g4 ) n! n! 10! × n Pn−10 10! n n! = ≥ 10 × ) = ∞, g4 = Ω(g5 ) limn→∞ ( n 20 n 20 10 + n 10 + n 10 10 10n + n20 10n limn→∞ ( ≥ n = 2.5n ) = ∞, g5 = Ω(g6 ) n 4 4 4n 4 n limn→∞ ( n = ( ) ) = ∞, g6 = Ω(g7 ) e e n en (lg n) lg lg n limn→∞ ( ≥ ) = ∞, g7 = Ω(g8 ) (lgn)! (lg n)lg n (lg n)! nlg lg n/e limn→∞ ( 5/2 = ) = ∞, g8 = Ω(g9 ) n n2 .5 n5/2 limn→∞ ( lg n = n5/2−lg 5 ) = ∞, g9 = Ω(g10 ) 5 5lg n nlg 5 limn→∞ ( 2 = 2 ) = ∞, g10 = Ω(g11 ) 5n + 7n 5n + 7n limn→∞ (

3

5n2 + 7n n =5 ) = ∞, g11 = Ω(g12 ) nln n lgn nln n nln n 2 limn→∞ ( = ) = , g12 = Θ(g13 ) lg(n!) nlg n − nln2 e lgn lg(n!) = ) = ∞, g13 = Ω(g14 ) limn→∞ ( 8n + 12 8 √ 8n + 12 limn→∞ ( √ = 8 n) = ∞, g14 = Ω(g15 ) √n n limn→∞ ( ) = ∞ (By p-test), g15 = Ω(g16 ) (lgn)2 (lgn)2 (lgn)2 ≥ ) = ∞, g16 = Ω(g17 ) limn→∞ ( ln ln n lgn ln ln ln 2 lnn limn→∞ ( = lg( ∗ )) = ∞, g17 = Ω(g18 ) ∗ lg(lg n) e lg n lg(lg ∗ n) lg(lg ∗ n) limn→∞ ( 1/lg n = ) = ∞, g18 = Ω(g19 ) n 2 Cluster point of Sequence 2 + sin n is [1, 3] Therefore cannot be classified; It cannot be written with omega notation with g9 through g11 . g12 and g13 ; g2 and g3 is in same partition since only difference between two is lower order term. limn→∞ (

4

4

Problem 4

Let f(n) and g(n) be asymptotically positive functions. Prove or disprove each of the following conjectures. Problem 1. f (n) = O(g(n)) implies lg(f (n)) = O(lg(g(n))) , where lg(g(n)) ≥ 1 and f (n) ≥ 1 for all sufficiently large n pf) For some c, n0 , ∀n ≥ n0 , f (n) ≤ cg(n) ∀n ≥ n0 , lg(f (n)) ≤ lg(c) + lg(g(n)) ≤ (1 + lg(c))lg(g(n)). There exists c0 = 1 + lg(c), n00 = n0 s. t. ∀n ≥ n00 lg(f (n)) ≤ c0 lg(g(n)) means lg(f (n)) = O(lg(g(n))). Problem 2. f (n) = O(g(n)) implies 2f (n) = O(2g(n) ) Counterexample) Let f (n) := 2lg n and g(n) := lg n. Then f (n) = O(g(n)). 2f (n) = n2 and 2g(n) = n Therefore f (n) ∈ / O(g(n)). Problem 3. f (n) = O((f (n))2 ) Counterexample) Let f (n) := 1/n Then (f (n))2 = 1/n2 Therefore, f (n) ∈ / O((f (n))2 ) Problem 4. f (n) = O(g(n)) implies g(n) = Ω(f (n)) pf) For some c, n0 , ∀n ≥ n0 , 0 ≤ f (n) ≤ cg(n). For some c0 = 1/c, n00 = n0 , ∀n ≥ n00 , 0 ≤ c0 f (n) ≤ g(n) which means g(n) = Ω(f (n)) Problem 5. f (n) + o(f (n)) = Θ(f (n)) pf) f (n) + o(f (n)) = 1+0 = 1. Therefore f (n)+o(f (n)) ∈ Θ(f (n)) f (n) which can be rewrote to f (n) + o(f (n)) = Θ(f (n)) limn−>∞

5

5

Problem 5

Solve the following recurrences by giving tight Θ-notation bounds. Justify your answers. Problem 6. T (n) = 2T (n/3) + nlg n pf) (Answer is T (n) = Θ(nlg n) ) T (n) ≥ nlg n therefore T (n) = Ω(nlg n). Claim) T (n) ≤ 3nlg n + 1 i) n = 1 T (n) can be considered as 1, therefore T (n) ≤ 3nlg n + 1 ii) n = 1 · · · k − 1 =⇒ n = k when k ≥ 2 T (n) = 2T (n/3) + nlg n ≤ 2 × 2n(lg n − lg3) + 2 + nlg n = 3nlg n − 2nlg3 + 2 < 3nlg n + 1 Therefore, T (n) = O(nlg n), T (n) = Θ(nlg n) Problem 7. T (n) = T (n − 2) + lgn T (n) = T (n − 2) + lgn = T (n − 4) + lg(n − 2) + lgn = · · · = T (n mod 2) + lg(n mod 2 + 2) + lg(n mod 2 + 4) + · · · + T (n) ≤ Θ(1) + nlgn = Ω(nlg n). And T (n mod 2) + lg(n mod 2 + 2) + lg(n mod 2 + 4) + · · · + T (n) > lg(n/2) ∗ (n/4 − 1) = O(nlgn). Thus It gives T (n) = Θ(nlg n). √ Problem 8. T (n) = T (n/2) + T ( n) + n pf) (Answer is T (n) = Θ(n)) T (n) ≥ n therefore T (n) = Ω(nn). Claim) T (n) ≤ 4n i) n ≤ 16 T (n) can be considered as 1 for small n, therefore T (n) ≤ 4n ii) n = 1 · · · k − 1 =⇒ n √= k when k ≥ 16 T (n) = T (n/2) + T ( n) + n ≤ T (n/2) + T (n/4) + n ≤ 2n + n + n = 4n Therefore, T (n) = O(n), T (n) = Θ(n) 6