Greedy Algorithms Mohan Kumar CSE@UTA CSE5311 Fall 2005

9/11/2005

CSE 5311 Fall 2005 M Kumar

1

The Greedy Principle • The problem: We are required to find a feasible solution that either maximizes or minimizes a given objective solution. • It is easy to determine a feasible solution but not necessarily an optimal solution. • The greedy method solves this problem in stages, at each stage, a decision is made considering inputs in an order determined by the selection procedure which may be based on an optimization measure. • The greedy algorithm always makes the choice that looks best at the moment. – For each decision point in the greedy algorithm, the choice that seems best at the moment is chosen • It makes a local optimal choice that may lead to a global optimal choice.

9/11/2005

CSE 5311 Fall 2005 M Kumar

2

1

Activity Selection Problem

• Scheduling a resource among several competing activities. • S = {1,2, 3, …, n} is the set of n proposed activities • The activities share a resource, which can be used by only one activity at a time -a Tennis Court, a Lecture Hall etc., • Each activity i has a start time, si and a finish time fi, where s i ≤ fi . • When selected, the activity takes place during time (si, fi) • Activities i and j are compatible if si ≥ fj or sj ≥ fi • The activity-selection problem selects the maximum-size set of mutually compatible activities • The input activities are in order by increasing finishing times. • f1 ≤ f2 ≤ f3 … ≤ fn ; Can be sorted in O (n log n ) time 9/11/2005

CSE 5311 Fall 2005 M Kumar

3

Procedure for activity selection (from CLRS)

Procedure GREEDY_ACTIVITY_SELECTOR(s, f) n ← length [S]; in order of increasing finishing times; A ← {1}; first job to finish j ← 1; for i ← 2 to n do if si ≥ fj then A ← A∪ {i}; j ← i;

9/11/2005

CSE 5311 Fall 2005 M Kumar

4

2

• i • 1

si 1

fi 4

• • •

• • • • • • • • • •

2 3 4 5 6 7 8 9 10 11

9/11/2005

3 0 5 3 5 6 8 8 2 12

5 6 7 8 9 10 11 12 13 14

• • • • • •

Initially we choose activity 1 as it has the least finish time. Then, activities 2 and 3 are not compatible as s2 < f1 and s3 < f1. We choose activity 4, s4 > f1, and add activity 4 to the set A. A = {1, 4} Activities 5, 6, and 7 are incompatible and activity 8 is chosen A = {1,4,8} Finally activity 10 is incompatible and activity 11 is chosen A {1,4,8,11} The algorithm can schedule a set of n activities in Θ (n) time.

CSE 5311 Fall 2005 M Kumar

5

Minimum-Cost Spanning Trees Consider a network of computers connected through bidirectional links. Each link is associated with a positive cost: the cost of sending a message on each link. This network can be represented by an undirected graph with positive costs on each edge. In bidirectional networks we can assume that the cost of sending a message on a link does not depend on the direction. Suppose we want to broadcast a message to all the computers from an arbitrary computer. The cost of the broadcast is the sum of the costs of links used to forward the message.

9/11/2005

CSE 5311 Fall 2005 M Kumar

6

3

Minimum-Cost Spanning Trees • Find a fixed connected subgraph, containing all the vertices such that the sum of the costs of the edges in the subgraph is minimum. This subgraph is a tree as it does not contain any cycles. • Such a tree is called the spanning tree since it spans the entire graph G. • A given graph may have more than one spanning tree • The minimum-cost spanning tree (MCST) is one whose edge weights add up to the least among all the spanning trees

9/11/2005

CSE 5311 Fall 2005 M Kumar

7

MCST

3

3 2

2

A

1 6

4

2 E

5

1

A Local Area Network

9/11/2005

A

3

3

F

1

2

B 6

4 5

1 D

F

3 1

2

B

2

C E 1

C

D

The equivalent Graph and the MCST

CSE 5311 Fall 2005 M Kumar

8

4

MCST • The Problem: Given an undirected connected weighted graph G =(V,E), find a spanning tree T of G of minimum cost. • Greedy Algorithm for finding the Minimum Spanning Tree of a Graph G =(V,E) The algorithm is also called Kruskal's algorithm. • At each step of the algorithm, one of several possible choices must be made, • The greedy strategy: make the choice that is the best at the moment

9/11/2005

CSE 5311 Fall 2005 M Kumar

9

Kruskal's Algorithm • • • • • • • • • • • • • • • • •

Procedure MCST_G(V,E) (Kruskal's Algorithm) Input: An undirected graph G(V,E) with a cost function c on the edges Output: T the minimum cost spanning tree for G T ← 0; VS ←0; for each vertex v ∈ V do VS = VS ∪ {v}; sort the edges of E in nondecreasing order of weight while |VS| > 1 do choose (v,w) an edge E of lowest cost; delete (v,w) from E; if v and w are in different sets W1 and W2 in VS do W1 = W1 ∪ W2; VS = VS - W2; T ← T∪ (v,w); return T

9/11/2005

CSE 5311 Fall 2005 M Kumar

10

5

Kruskals_ MCST • The algorithm maintains a collection VS of disjoint sets of vertices • Each set W in VS represents a connected set of vertices forming a spanning tree • Initially, each vertex is in a set by itself in VS • Edges are chosen from E in order of increasing cost, we consider each edge (v, w) in turn; v, w ∈ V. • If v and w are already in the same set (say W) of VS, we discard the edge • If v and w are in distinct sets W1 and W2 (meaning v and/or w not in T) we merge W1 with W2 and add (v, w) to T.

9/11/2005

CSE 5311 Fall 2005 M Kumar

11

Kruskals_ MCST Consider the example graph shown earlier, The edges in nondecreasing order [(A,D),1],[(C,D),1],[(C,F),2],[(E,F),2],[(A,F),3],[( A,B),3], [(B,E),4],[(D,E),5],[(B,C),6] EdgeActionSets in VSSpanning Tree, T =[{A},{B},{C},{D},{E},{F}]{0}(A,D)merge

A

[{A,C,D}, {B}, {E}, {F}] {(A,D), (C,D)} (C,F) merge [{A,C,D,F},{B},{E}]{(A,D),(C,D), (C,F)} (E,F) merge [{A,C,D,E,F},{B}]{(A,D),(C,D), (C,F),(E,F)}(A,F) reject

2 E

A

3

3 2

F

[{A,D}, {B},{C}, {E}, {F}] {(A,D)} (C,D) merge

1

B 6

4 5

1 D

C

F

3 1

2

B

2 E 1

C

D

The equivalent Graph and the MCST

[{A,C,D,E,F},{B}]{(A,D),(C,D), (C,F), (E,F)}(A,B) merge [{A,B,C,D,E,F}]{(A,D),(A,B),(C,D), (C,F),(E,F)}(B,E) reject (D,E) reject (B,C) reject 9/11/2005

CSE 5311 Fall 2005 M Kumar

12

6

Kruskals_ MCST Complexity • • • • • • •

Steps 1 thru 4 take time O (V) Step 5 sorts the edges in nondecreasing order in O (E log E ) time Steps 6 through 13 take O (E) time The total time for the algorithm is therefore given by O (E log E) The edges can be maintained in a heap data structure with the property, E[PARENT(i)] ≤ E[i] remember, this property is the opposite of the one used in the heapsort algorithm earlier. This property can be used to sort data elements in nonincreasing order. Construct a heap of the edge weights, the edge with lowest cost is at the root During each step of edge removal, delete the root (minimum element) from the heap and rearrange the heap. The use of heap data structure reduces the time taken because at every step we are only picking up the minimum or root element rather than sorting the edge weights.

• • •

9/11/2005

CSE 5311 Fall 2005 M Kumar

13

Prim’s Algorithm A

A A F 2

3

3

1

2

1

1

D

D

6

4

E

B

5

1

1

C

C

D A

The equivalent Graph and the MCST A F

2

F 1 2

2

1

1

E 1 9/11/2005

D

CSE 5311 Fall 2005 M Kumar

B

1 2

E

2

C

1 D

3

F

A

C

C

D 14

7

Huffman codes Huffman codes are used to compress data. We will study Huffman's greedy algorithm for encoding compressed data.

Data Compression • •

• •

A given file can be considered as a string of characters. The work involved in compressing and uncompressing should justify the savings in terms of savings in storage area and/or communication costs. In ASCII all characters are represented by bit strings of size 7. For example if we had 100000 characters in a file then we need 700000 bits to store the file using ASCII. 9/11/2005

CSE 5311 Fall 2005 M Kumar

15

Example The file consists of only 6 characters as shown in the table below. Using the fixed-length binary code, the whole file can be encoded in 300,000 bits. However using the variable-length code , the file can be encoded in 224,000 bits.

Frequency (in thousands) Fixed-length codeword Variable-length codeword

a 45

b 13

c 12

d 16

e 9

f 5

000

001

010

011

100

101

0

101

100

111

1101 1100

A variable length coding scheme assigns frequent characters, short code words and infrequent characters, long words. In the above variable-length code, 1-bit string represents the most frequent character a, and a 4-bit string represents the most infrequent character f. 9/11/2005 CSE 5311 Fall 2005 16 M Kumar

8

Let us denote the characters by C1, C2, …, Cn and denote their frequencies by f1, f2, ,,,, fn. Suppose there is an encoding E in which a bit string Si of length si represents Ci, the length of the file compressed by using encoding E is n

L( E , F ) = ∑ si ⋅ f i i =1

9/11/2005

CSE 5311 Fall 2005 M Kumar

17

Prefix Codes •

• • • •

The prefixes of an encoding of one character must not be equal to a complete encoding of another character. •1100 and 11001 are not valid codes •because 1100 is a prefix of 11001 This constraint is called the prefix constraint. Codes in which no codeword is also a prefix of some other code word are called prefix codes. Shortening the encoding of one character may lengthen the encodings of others. To find an encoding E that satisfies the prefix constraint and minimizes L(E,F).

9/11/2005

CSE 5311 Fall 2005 M Kumar

18

9

0

101

100

111

The prefix code for file can be represented by a binary tree in which every non leaf node has two children. Consider the variable-length code of the table above, a tree corresponding to the variable-length code of the table is shown below. Note that the length of the code for a character is equal to the depth of the character in the tree shown. 9/11/2005

1101

1100

1

100

0

55

a:45 5 0 c:12

1

0 25

30

1

0

b:13

14

1 d:16

0

1

f:5

e:9

CSE 5311 Fall 2005 M Kumar

19

Greedy Algorithm for Constructing a Huffman Code The algorithm builds the tree corresponding to the optimal code in a bottom-up manner. The algorithm begins with a set of C leaves and performs a sequence of 'merging' operations to create the tree. C is the set of characters in the alphabet.

9/11/2005

CSE 5311 Fall 2005 M Kumar

20

10

Procedure Huffman_Encoding(S,f); Input : S (a string of characters) and f (an array of frequencies). Output : T (the Huffman tree for S) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

insert all characters into a heap H according to their frequencies; while H is not empty do if H contains only one character x then x ← root (T); else z ← ALLOCATE_NODE(); x ← left[T,z] ← EXTRACT_MIN(H); y ← right[T,z] ← EXTRACT_MIN(H); f z ← f x + f y; INSERT(H,z);

9/11/2005

f:5

CSE 5311 Fall 2005 M Kumar

e:9

c:12

b:13

21

d:16

a:45

•The algorithm is based on a reduction of a problem with n characters to a problem with n-1 characters. •A new character replaces two existing ones. c:12

b:13

14

0 f:5

0 f:5 9/11/2005

14

1 e:9

d:16

1

a:45

e:9

d:16

0

25

c:12 CSE 5311 Fall 2005 M Kumar

1

a:45

b:13 22

11

0

25

c:12

1

30

0

b:13

14

0

1

f:5

a:45

1 d:16

e:9

Suppose Ci and Cj are two characters with minimal frequency, there exists a tree that minimizes L (E,F) in which these characters correspond to leaves with the maximal distance from the root.

a:45 0

25

b:13

100

f:5

55

0 25

c:12

9/11/2005

101

100

111

1

d:16

e:9

23

1

1

0

b:13

14

0 f:5

0

14

0

1

1

a:45

0

30

0

CSE 5311 Fall 2005 M Kumar

0

1

1

c:12 9/11/2005

55

0

1101

30

1 d:16

1 e:9

1100

CSE 5311 Fall 2005 M Kumar

24

12

Complexity of the algorithm Building a heap in step 1 takes O(n) time Insertions (steps 7 and 8) and deletions (step 10) on H take O (log n) time each Therefore Steps 2 through 10 take O (n logn) time Thus the overall complexity of the algorithm is O( n logn ).

9/11/2005

CSE 5311 Fall 2005 M Kumar

25

Exercise Problems •

•

•

• •

Let G = (V,E) be a connected undirected weighted graph. Assume for simplicity that the weights are positive and distinct. Let e be an edge of G. Denote by T(e) the spanning tree of G that has minimum cost among all spanning trees of G that contains e. Design an algorithm to find T(e) for all edges e ∈ E. The algorithm should run in time O(V2). An Euler circuit of an undirected graph G(V,E) is a path that starts and ends at the same node and contains each edge of G exactly once. – Show that a connected, undirected graph has an Euler circuit if and only if each node is of even degree. – Let G (V,E) be an undirected graph with m edges in which every node is of even degree. Give an O(V) algorithm to construct an Euler circuit for G. Suppose that we have a set of k activities to schedule among n number of lecture halls; activity i starts at time si and terminates at time fi 1 ≤ i ≤ k. We wish to schedule all activities using as few lecture halls as possible. Give an efficient greedy algorithm to determine which activity should use which lecture hall. Write an algorithm to find the biconnected components and articulation points of a given connected undirected graph G. Given a connected undirected graph G = (V,E), and three edges, a, b, and c, find whether there exists a cycle in G that contains both a and b but does not contain c. The algorithm should run in linear time.

9/11/2005

CSE 5311 Fall 2005 M Kumar

26

13

• The fractional knapsack problem • limited supply of each item • Each item has a size and a value per unit (e.g., pound)

– greedy strategy • Compute value per pound for each item • Arrange these in nondecreasing order • Fill sack with the item of greatest value per pound until either the item is exhausted or the sack is full • If sack is not full, fill the remainder with the next item in the list • Repeat until sack is full How about a 0-1 Knapsack?? Can we use greedy strategy?

9/11/2005

CSE 5311 Fall 2005 M Kumar

27

Dynamic programming techniques

Topics •Basics of DP •Matrix-chain Multiplication •Longest Common subsequence • All-pairs Shortest paths

Further Reading Chapter 6 Textbook

9/11/2005

CSE 5311 Fall 2005 M Kumar

28

14

Dynamic programming •Solves problems by combining the solutions to subproblems •DP is applicable when subproblems are not independent Subproblems share subsubproblems In such cases a simple Divide and Conquer strategy solves common subsubproblems. •In DP every subproblem is solved just once and the solution is saved in a table for future reference (avoids re-computation). •DP is typically applied to optimization problems •A given problem may have many solutions, DP chooses the optimal solution. 9/11/2005

CSE 5311 Fall 2005 M Kumar

29

Four stages of Dynamic Programming

♦Characterize the structure of an optimal solution ♦Recursively define the value of an optimal solution ♦Compute the value of an optimal solution in a bottom-up fashion ♦Construct an optimal solution from computed results 9/11/2005

CSE 5311 Fall 2005 M Kumar

30

15

Longest common subsequence A subsequence is formed from a list by deleting zero or more elements (the remaining elements are in order) A common subsequence of two lists is a subsequence of both. The longest common subsequence (LCS) of two lists is the longest among the common subsequences of the two lists. Example: abcabba and cbabac are two sequences baba is a subsequence of both 9/11/2005

CSE 5311 Fall 2005 M Kumar

a

9/11/2005

b

c

a

b

b

a

b

a

c

b

a

b

CSE 5311 Fall 2005 M Kumar

31

b

a

a

c

32

16

To find the length of an LCS of lists x and y, we need to find the lengths of the LCSs of all pairs of prefixes. a prefix is an initial sublist of a list If x = (a1,a2,a3, . . ., am) and y = (b1,b2,b3, . . ., bn) 0 ≤ i ≤ m and 0≤ j≤ n Consider an LCS of the prefix (a1,a2,a3, . . ., ai) from x and of the prefix (b1,b2,b3, . . ., bj) from y. If i or j = 0 then one of the prefixes is ε and the only possible common subsequence between x and y is ε and the length of the LCS is zero. 9/11/2005

CSE 5311 Fall 2005 M Kumar

33

L(i,j) is the length of the LCS of (a1,a2,a3, . . ., ai) and (b1,b2,b3, . . ., bj). BASIS: If i+j = 0, then both i and j are zero and so the LCS is ε. INDUCTION: Consider i and j, and suppose we have already computed L(g,h) for any g and h such that g+h < i+j. 1.If either i or j is 0 then L(i,j) = 0. 2.If i>0 and j>0, and ai≠ bj then L(i,j) = max(L(i,j-1),L(i-1,j)). 3.If i >0 and j> 0, and ai = bj then L(i,j) = L(i-1,j-1)+1. 9/11/2005

CSE 5311 Fall 2005 M Kumar

34

17

ε a c a b

1.If either i or j is 0 then L(i,j) = 0. 2.If i>0 and j>0, and ai≠ bj then L(i,j) = max(L(i,j-1),L(i-1,j)). 3.If i >0 and j> 0, and ai = bj then L(i,j) = L(i-1,j-1)+1.

9/11/2005

0 0 0 ε

1 1 1 1 0 0 ab

2 1 0 c

2 1 0 a

CSE 5311 Fall 2005 M Kumar

35

Procedure LCS(x,y) Input : The lists x and y Output : The longest common subsequence and it's length 1. for j ← 0 to n do 2. L[0,j] ← 0; 3. for i ← 1 to m do 4. L[i,0] ←0; 5. for j ← 1 to n do 6. if a[i] ≠ b[j] then 7. L[i,j] ← max {L[i-1,j],L[i,j-1]}; 8. else 9 L[i,j] ← 1+L[i-1,j-1]; 9/11/2005

CSE 5311 Fall 2005 M Kumar

36

18

Example: Consider, lists x = abcabba and y = cbabac

c60 a50 b4 0 a30 b2 0 c10 000 0

1 1 1 1 0 0 0 a

2 2 2 1 1 0 0 b

3 2 2 1 1 1 0 c

9/11/2005

3 3 3 4 3 3 3 4 2 3 3 3 2 2 2 3 1 2 2 2 1 1 1 1 0 0 0 0 a b b a CSE 5311 Fall 2005 M Kumar

37

Consider another example abaacbacab and bacabbcaba LCS : bacacab

b a c a b c a a b a 0

0 0 0 0 0 0 0 0 0 0 0 0 9/11/2005

1 1 1 1 1 1 1 1 1 0 0 b

2 2 2 2 2 2 2 2 1 1 0 a

3 3 3 3 3 3 2 2 1 1 0 c

4 4 4 4 3 3 3 2 1 1 0 a

5 4 4 4 4 3 3 2 2 1 0 b

5 4 4 4 4 3 3 2 2 1 0 b

CSE 5311 Fall 2005 M Kumar

5 5 5 4 4 4 3 2 2 1 0 c

6 6 5 5 4 4 3 3 2 1 0 a

7 6 5 5 5 4 3 3 2 1 0 b

7 6 6 6 5 4 4 3 2 1` 0 a 38

19

Matrix-chain Multiplication Consider the matrix multiplication procedure 1. 2. 3. 4. 5. 6. 7. 8.

MATRIX_MULTIPLY(A,B) if columns[A] ≠ rows[B] then error "incompatible dimensions” else for i ← 1 to rows[A] do for j ←1 to columns[B] do C[i,j] ←0; for k ← 1 to columns [A] do C[i,j] ← C[i,j]+A[i,k]*B[k,j]; return C

9/11/2005

CSE 5311 Fall 2005 M Kumar

39

The time to compute a matrix product is dominated by the number of scalar multiplications in line 7. If matrix A is of size (p×q) and B is of size (q×r), then the time to compute the product matrix is given by pqr. Consider three matrices A1, A2, and A3 whose dimensions are respectively (10×100), (100×5), (5×50). Now there are two ways to parenthesize these multiplications I ((A1×A2) ×A3) II (A1× (A2×A3)) 9/11/2005

CSE 5311 Fall 2005 M Kumar

40

20

First Parenthesization Product A1×A2 requires 10×100×5 = 5000 scalar multiplications A1×A2 is a (10×5) matrix (A1×A2) ×A3 requires 10 × 5 × 50 = 2500 scalar multiplications. Total : 7,500 multiplications       

10×100

      

A1 10×5

      ×      

      ×     2  

A1×A

9/11/2005

100 ×5 A2 5 ×50 A3

      =      

      2  

10 ×5 A1×A

      =   (A ×A ) 1 2   ×A3  

10 ×50

A1

      

CSE 5311 Fall 2005 M Kumar

41

Second Parenthesization Product A2×A3 requires 100×5×50 = 25,000 scalar multiplications A2×A3 is a (100×50) matrix A1× (A2×A3) requires 10×100×50 = 50,000 scalar multiplications Total : 75,000 multiplications.              

9/11/2005

100×5 A2

      ×      

      =      

100 ×50

5 ×50 A3

 

 

×      

=     3    

A2×A3

      

     A1×  (A2 ×A3)  

10×100   100 ×50   10 ×50 A1

A2×A

CSE 5311 Fall 2005 M Kumar

The first parenthesization is 10 times faster than the second one!! How to pick the best parenthesization ? 42

21

The matrix-chain matrix multiplication Given a chain (A1, A2, . . . ,An) of n matrices, where for i = 1,2, …, n matrix Ai has dimension pi-1×pi, fully parenthesize the product A1A2…An in a way that minimizes the number of scalar multiplications. The order in which these matrices are multiplied together can have a significant effect on the total number of operations required to evaluate the An optimal solution to an instance of a matrix--chain product. multiplication problem contains within it optimal solutions to the subproblem instances.

9/11/2005

CSE 5311 Fall 2005 M Kumar

43

Let, P(n) : The number of alternative parenthesizations of a sequence of n matrices We can split a sequence of n matrices between kth and (k+1)st matrices for any k = 1, 2, …, n-1 and we can then parenthesize the two resulting subsequences independently,

1 if n = 1  P(n) =  n−1  ∑ P(k) ⋅ P(n − k) if n ≥ 2 k =1

This is an exponential in n 9/11/2005

CSE 5311 Fall 2005 M Kumar

44

22

Consider A1×A2 ×A3 ×A4 if k =1, then A1× (A2 ×(A3 ×A4)) or A1×((A2 ×A3 )×A4) if k =2 then (A1×A2) × (A3 ×A4) if k =3 then ((A1×A2) ×A3) ×A4 or (A1×(A2 ×A3)) ×A4 9/11/2005

CSE 5311 Fall 2005 M Kumar

45

Structure of the Optimal Parenthesization A i..j = Ai×Ai+1× . . . × Aj An optimal parenthesization splits the product Ai..j = (Ai×Ai+1× . . . × Ak) × (Ak+1×Ak+2× . . . × Aj) for 1≤ k < n The total cost of computing Ai..j = cost of computing (Ai×Ai+1× . . . × Ak)

+ cost of computing (Ak+1× Ak+2× . . . × Aj) + cost of multiplying the matrices Ai..k and Ak+1..j.

Ai...k must also be optimal if we want Ai..j to be optimal. If Ai..k is not optimal then Ai..j is not optimal. Similarly Ak+1..j must also be optimal. 9/11/2005

CSE 5311 Fall 2005 M Kumar

46

23

Recursive Solution We'll define the value of an optimal solution recursively in terms of the optimal solutions to subproblems. m[i,j] = minimum number of scalar multiplications needed to compute the matrix Ai..j m[1,n] = minimum number of scalar multiplications needed to compute the matrix A1..n. If i = j ; the chain consists of just one matrix A i..i = Ai - no scalar multiplications m[i,i] = 0 for i = 1, 2, …, n. m[i,j] = minimum cost of computing the subproducts Ai..k and A k+1 ..j + cost of multiplying these two matrices Multiplying Ai..k and Ak+1....j takes pi-1pk pj scalar multiplications m[i,j] = m[i,k] + m[k+1,j] + pi-1pk pj for i≤ k < j 9/11/2005

CSE 5311 Fall 2005 M Kumar

47

The optimal parenthesization must use one of these values for k, we need to check them all to find the best solution. Therefore,

 0 if i = j  m[i, j] =  min i ≤ k < j {m[i, k] + m[k + 1, j]} + pi −1 pk p j  Let s[i,j] be the value of k at which we can split the product Ai× Ai+1× . . . × Aj to obtain the optimal parenthesization. s[i,j] equals a value of k such that m[i,j] = m[i,k] + m[k+1,j] + pi-1pk pj for i≤ k < j 9/11/2005

CSE 5311 Fall 2005 M Kumar

48

24

Procedure Matrix_Chain_Order (p) Input: sequence (p0,p1,…pn) Output : an auxiliary table m[1..n,1..n] with m[i,j] costs and another auxiliary table s[1..n,1..n] with records of index k which achieves optimal cost in computing m[i,j] 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 9/11/2005 13.

n←length[p]-1; for i ← 1 to n do m[i,i] ←0; for l ← 2 to n do for i ← 1 to n-l+1 do j ← i+l-1 m[i,j]←∞; for k ← i to j-1 do q ←m[i,k]+m[k+1,j]+pi-1pkpj; if q < m[i,j]; then m[i,j] ←q; s[i,j] ←k; CSE 5311 Fall 2005 return m and s M Kumar

49

Consider A1×A2 ×A3 ×A4 Consider Four Matrices A1 : 10 × 20

A2 : 20×50

A3: 50×1

A4 : 1 ×100

1 0 10,000 1200 2200

2 -0 1000 3000

3 --0 5000

A1×((A2 ×A3 )×A4) if k =2 then (A1×A2) × (A3 ×A4)

MIN[(10,000+500),(1000+200) j↓ i→ 1 2 3 4

if k =1, then A1× (A2 ×(A3 ×A4))

4 ---0

if k =3 then ((A1×A2) ×A3) ×A4 10 ×50 and (A1×(A2 ×A3)) ×A4 20 ×1

9/11/2005

CSE 5311 Fall+2005 m[i,j] = m[i,k] + m[k+1,j] pi-1pk pj for i≤ k < j

50

M Kumar

25

Consider, A1 (30×35)A2 (35×15)A3 (15×5), A4(5×10), A5(10×20), A6(20×25)

m

1 0 15,750 7,875 9,375 11,875 15,125

j↓/i→ 1 2 3 4 5 6

j↓/i→

s

2 3 4 5 6

1 1 1 3 3 3

2 2 3 3 3

9/11/2005

2 -0 2,625 4,375 7,125 10,500

3 3 3 3

3 --0 750 2,500 5,375

4 4 5

4 ---0 1,000 3,500

5 5

5 ----0 5,000

6 -----0

(A1..A3)× (A4..A6) (A1 × (A2 × A3)) × ((A4 × A5) × A6)

CSE 5311 Fall 2005 M Kumar

51

Complexity? With and without DP? • T(1) ≥ 1 • T(n) ≥ 1 +

n−1

∑T (k) + T (n − k) +1) for n ≥ 1 k =1

• T(n) ≥ • Exponential in n n −1

2 ∑ T (i ) + n i =1

9/11/2005

CSE 5311 Fall 2005 M Kumar

52

26

• Give a dynamic-programming solution to the 0-1 knapsack problem that runs in O(nW) time, where n is the number of items and W is the maximum weight of items that the thief can put in his knapsack. The weight is measured in Kgs (say). The maximum weight is an integer. Let S be the optimal solution for W And i be the highest numbered item in S, the items are 1..n. S’ = S- {i} is an optimal solution for W-wi. Kilos and items 1.. i-1. The value of the solution is S is the value vi of item i and the value of the solution S’. Let c[i,w] be the value of the solution for items 1..i and maximum weight w. 0 if i =0 or w=0.  c[i-1,w] if wi > w max (vi+c[i-1,w-wi],c[i-1,w] if i >0 and w ≥ wi

C[i,w] =

The value of the solution for i items either includes item i, in which case it is vi plus a Subproblem solution for i-1 items and the weight excluding wi or doesn’t include the item i. Inputs : W, n, v= and w = < w1, w2, …, wn> The table is c[0..n, 0..W] – each entry is referred to as c[i,j] The first row entries are filled first and then the second entries are computed and so on (very similar to the LCS solution). At the end c[n,W] contains the maximum value. Trace the items which are part of the solution from c[n,W]. If c[i,w] = c[i-1,w] then i is not part of the solution, go to c[i-1,w] and trace back If c[i,w] ≠ c[i-1,w] then I is not part of the solution, trace with c[i-1,w-wi]. 9/11/2005

CSE 5311 Fall 2005 M Kumar

53

Consider the problem of neatly printing a paragraph on a printer. The input text is a sequence of n words of length l1,l2, . . ., ln, measured in input characters. We want to print this paragraph neatly on a number of lines that hold a maximum of M characters each. Our criterion of "neatness" is as follows. If a given line contains words i through j and leave exactly one space between words, the number of extra space characters at the end of the line is j

M − j + i − ∑lk k =i

We wish to minimize the sum, over all the lines except the last of the extra space characters at the ends of lines. Give a dynamic programming algorithm to print a paragraph of n words neatly on a printer. Analyze the running time and space requirements of your algorithm.

9/11/2005

CSE 5311 Fall 2005 M Kumar

54

27

Hints • Assume that no word is longer than a line • Determine the cost of a line containing words i through j (cost is the number of free spaces) • We want to minimize the sum of line costs over all lines in the paragraph. • Try to represent the above by a recursive expression

9/11/2005

CSE 5311 Fall 2005 M Kumar

55

• A ski rental agency has m pairs of skis, where the height of the ith pair of skis is si. There are n skiers who wish to rent skis, where the height of the ith skier is hi. Ideally, each skier should obtain a pair of skis whose height matches with his own height as closely as possible. Design an efficient algorithm to assign skis so that the sum of the absolute differences of the heights of each skier and his/her skis is minimized.

9/11/2005

CSE 5311 Fall 2005 M Kumar

56

28