Dynamic Programming( Matrix Multiplication)

Dynamic Programming( Matrix Multiplication) Arash Rafiey 21 January 2016 Arash Rafiey Dynamic Programming( Matrix Multiplication) Matrix Multipli...
30 downloads 0 Views 295KB Size
Dynamic Programming( Matrix Multiplication) Arash Rafiey

21 January 2016

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

Matrix Multiplications Given: a “chain” of matrices (A1 , A2 , . . . An ), with Ai having dimension di−1 × di . Goal: compute the product A1 · A2 · · · An as fast as possible

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

Matrix Multiplications Given: a “chain” of matrices (A1 , A2 , . . . An ), with Ai having dimension di−1 × di . Goal: compute the product A1 · A2 · · · An as fast as possible Clearly, time to multiply two matrices depends on dimensions Does the order of multiplication (= parenthesization) matter? Example: n = 4. Possible orders: (A1 (A2 (A3 A4 ))) (A1 ((A2 A3 )A4 )) ((A1 A2 )(A3 A4 )) ((A1 (A2 A3 ))A4 ) (((A1 A2 )A3 )A4 )

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

Suppose A1 is 10 × 100, A2 is 100 × 5, A3 is 5 × 50, and A4 is 50 × 10 Assume that multiplication of a (p × q)-matrix and a (q × r )-matrix takes pqr steps (a straightforward algorithm)

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

Suppose A1 is 10 × 100, A2 is 100 × 5, A3 is 5 × 50, and A4 is 50 × 10 Assume that multiplication of a (p × q)-matrix and a (q × r )-matrix takes pqr steps (a straightforward algorithm) Order 2: (A1 ((A2 A3 )A4 )) 100 · 5 · 50 + 100 · 50 · 10 + 10 · 100 · 10 = 85, 000 Order 5: (((A1 A2 )A3 )A4 ) 10 · 100 · 5 + 10 · 5 · 50 + 10 · 50 · 10 = 12, 500 Seems it might be a good idea to find a “good” order

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

How many orders are there? Can we just check all of them? ( we look only at fully parenthesized matrix products)

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

How many orders are there? Can we just check all of them? ( we look only at fully parenthesized matrix products) Let P(n) be the number of orders of a sequence of n matrices Clear, P(1) = 1 (only one matrix)

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

How many orders are there? Can we just check all of them? ( we look only at fully parenthesized matrix products) Let P(n) be the number of orders of a sequence of n matrices Clear, P(1) = 1 (only one matrix) If n ≥ 2, a matrix product is the product of two matrix sub-products. Split may occur between k-th and (k + 1)-st position, for any k = 1, 2, . . . , n − 1 (“top-level multiplication”)

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

How many orders are there? Can we just check all of them? ( we look only at fully parenthesized matrix products) Let P(n) be the number of orders of a sequence of n matrices Clear, P(1) = 1 (only one matrix) If n ≥ 2, a matrix product is the product of two matrix sub-products. Split may occur between k-th and (k + 1)-st position, for any k = 1, 2, . . . , n − 1 (“top-level multiplication”) Thus  1 if n = 1 P(n) = Pn−1 k=1 P(k) · P(n − k) if n ≥ 2  P(n) = n1 2n−2 n−1

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

How many orders are there? Can we just check all of them? ( we look only at fully parenthesized matrix products) Let P(n) be the number of orders of a sequence of n matrices Clear, P(1) = 1 (only one matrix) If n ≥ 2, a matrix product is the product of two matrix sub-products. Split may occur between k-th and (k + 1)-st position, for any k = 1, 2, . . . , n − 1 (“top-level multiplication”) Thus  1 if n = 1 P(n) = Pn−1 k=1 P(k) · P(n − k) if n ≥ 2  P(n) = n1 2n−2 n−1 Unfortunately, P(n) = Ω(4n /n3/2 ), and thus (easier to see) P(n) = Ω(2n ) Thus “brute-force approach” (check all parenthesization) is no good Arash Rafiey

Dynamic Programming( Matrix Multiplication)

We will use the Dynamic programming approach to optimally solve this problem. The four basic steps when designing Dynamic programming algorithm: 1 2 3

4

Characterize the structure of an optimal solution Recursively define the value of an optimal solution Compute the value of an optimal solution in a bottom-up fashion Construct an optimal solution from computed information

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

1. Characterizing structure

Let Ai,j = Ai · · · Aj for i ≤ j. If i < j, then any parenthesization of Ai,j must split product at some k, i ≤ k < j, i.e., compute Ai,k , Ak+1,j , and then Ai,k · Ak+1,j .

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

1. Characterizing structure

Let Ai,j = Ai · · · Aj for i ≤ j. If i < j, then any parenthesization of Ai,j must split product at some k, i ≤ k < j, i.e., compute Ai,k , Ak+1,j , and then Ai,k · Ak+1,j . Hence, for some k, the cost of computing Ai,j is the cost of computing Ai,k plus the cost of computing Ak+1,j plus the cost of multiplying Ai,k and Ak+1,j .

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

Optimal substructure: Suppose that optimal parenthesization of Ai,j splits the product between Ak and Ak+1 . Then, parenthesizations of Ai,k and Ak+1,j within this optimal parenthesization must be also optimal (otherwise, substitute the opt. parenthesization of Ai,k (resp. Ak+1,j ) to current parenthesization of Ai,j and obtain a better solution — contradiction) Use optimal substructure to construct an optimal solution: 1 2 3

split into two subproblems (choosing an optimal split), find optimal solutions to subproblem, combine optimal subproblem solutions.

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

A recursive solution Let m[i, j] denote minimum number of multiplications needed to compute Ai,j = Ai · Ai+1 · · · Aj (full problem: m[1, n]). Recursive definition of m[i, j]: if i = j, then m[i, j] = m[i, i] = 0 (no multiplication needed)

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

A recursive solution Let m[i, j] denote minimum number of multiplications needed to compute Ai,j = Ai · Ai+1 · · · Aj (full problem: m[1, n]). Recursive definition of m[i, j]: if i = j, then m[i, j] = m[i, i] = 0 (no multiplication needed) if i < j, assume optimal split at k, i ≤ k < j. Since each matrix Ai is di−1 × di , Ai,k is di−1 × dk and Ak+1,j is dk × dj , m[i, j] = m[i, k] + m[k + 1, j] + di−1 · dk · dj

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

A recursive solution Let m[i, j] denote minimum number of multiplications needed to compute Ai,j = Ai · Ai+1 · · · Aj (full problem: m[1, n]). Recursive definition of m[i, j]: if i = j, then m[i, j] = m[i, i] = 0 (no multiplication needed) if i < j, assume optimal split at k, i ≤ k < j. Since each matrix Ai is di−1 × di , Ai,k is di−1 × dk and Ak+1,j is dk × dj , m[i, j] = m[i, k] + m[k + 1, j] + di−1 · dk · dj We do not know optimal value of k. There are j − i possibilities, k = i, i + 1, . . . , j − 1, hence  if i = j  0 mini≤k k − 2. Now z1 Z 0 zk is a palindrome subsequence of xi , xi+1 , . . . , xj and longer than Z , a contradiction. Arash Rafiey

Dynamic Programming( Matrix Multiplication)

Case 2. z1 6= xi . We have z1 = x` , ` > i and hence Z is a longest palindrome subsequence of x` , x`+1 , . . . , xj . Since x` , x`+1 , . . . , xj is a subsequence of xi+1 , . . . , xj , Z is a longest palindrome subsequence of xi+1 , xi+2 , . . . , xj . Case 3. zk 6= xj . Similar to (2).

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

Now we design an algorithm based on the Lemma. Let S = x1 , x2 , . . . , xn . Let M[i, j] be the length of a longest palindrome subsequence from xi to xj . If i = j then M[i, i] = 1 (Since xi by itself is a palindrome). Our goal is to compute M[1, n].

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

Now we design an algorithm based on the Lemma. Let S = x1 , x2 , . . . , xn . Let M[i, j] be the length of a longest palindrome subsequence from xi to xj . If i = j then M[i, i] = 1 (Since xi by itself is a palindrome). Our goal is to compute M[1, n]. Recursive formula : If xi = xj then M[i, j] = M[i + 1, j − 1] + 2 otherwise : M[i, j] = max{M[i + 1, j], M[i, j − 1]}

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

Longest-palindrome-subsequence(S) 1. int n = length(S); // Initializing M. 2. for (i = 1 to n) M[i, i] = 1; // by definition 3. for (j = 1 to n − 1 ) 4.

for (i = 1 to n − j)

5.

if (S[i] = S[i + j])

6. 7.

M[i, i + j] = M[i + 1, i + j − 1] + 2 else

8.

M[i, i + j] = max{M[i + 1, i + j], M[i, i + j − 1]}

Arash Rafiey

Dynamic Programming( Matrix Multiplication)

Suggest Documents