Induction and Recursion

Induction and Recursion CSC 326 by Matt Bækgaard Pedersen ([email protected]) recursion: noun. See recursion. [New Hacker’s Dictionary] In this ha...
Author: Gordon Hensley
11 downloads 1 Views 135KB Size
Induction and Recursion CSC 326 by

Matt Bækgaard Pedersen

([email protected])

recursion: noun. See recursion. [New Hacker’s Dictionary]

In this handout I will try to show some examples of induction and recursion, and how induction and recursion are tied together. I will try to show how an induction proof is easily turned into a recursive function written in C++, and how a C++ function can be proven correct using induction.

1

Induction 101

Let P (n) be some statement depending on a value n. To prove ∀ n ≥ 0 : P (n) by induction we need to prove the following two things: 1. (The base case) A proof for P (0). 2. (The induction step) A proof for P (n) assuming P (n − 1) holds (P (n − 1) ⇒ P (n)).

2

A simple example

Show that the following equality holds for all positive values of n:

s(n) =

n X

i=

i=1

n(n + 1) 2

To prove this by induction we must first establish a base case: Base case: Show s(1) = 1: s(1) =

1 X

i=1=

i=1

1(1 + 1) 2

We now must set up an induction hypothesis before turning to the induction step.

1

Induction hypothesis: Assume ∀ k : 1 ≤ k ≤ n − 1 that s(k) =

k X

i=

i=1

k(k + 1) 2

We must now show that this also holds for a value n based on the induction hypothesis. Induction step: We wish to show that s(n) =

n X

i=

i=1

s(n) = =

n X

i

i=1 n−1 X

n(n + 1) 2

Definition of n. !

i +n

Decompose problem.

i=1

(n − 1)(n − 1 + 1) +n 2 (n − 1)n 2n = + 2 2 n(n + 1) = 2

=

Induction hypothesis.

We can now use the information from the induction proof to construct a C++ function to calculate s(n). Remember that a recursive function (let us call it f) in general looks like this: f () { if (base case test) { return ; else { return f (reduced ) (local computation on parts of ); }

2

This shows that you must determine the following things in order to successfully write a recursive function: 1. What is the base case test, i.e., what determines when you should no longer call recursively. 2. What is the base case value that is returned when the base case test evaluates to true. This is typically equivalent to the base case of the induction proof as well. 3. How do you break the problem into smaller pieces: you must break the problem into a part that you can call recursively on and a part that you can handle locally (or you break the problem into 2 or more parts on which you call recursively). A smaller problem passed to the procedure recursively is equivalent to using the induction hypothesis on a smaller problem. 4. Call recursively on the reduced problem. 5. Perform a local computation on the part of the problem that was not passed on to the recursive call. 6. How to combine the result from the recursive call with the local computation to yield a complete result. This is the value that is returned if you are not in the base case. If you can identify these 6 points for each problem it is not difficult to convert them into a recursive function using the above skeleton. Let us do that for s(n). Assume that we have the following header of the s function that we are going to implement in C++: int s (int n) Base case test: According to the induction proof the base case is when n = 1, so the base case test should be ’if (n == 1)’. Base case value: According to the induction proof the base case value is 1 (this should be fairly straight forward too - remembering that the sum of the 3

first 1 numbers, starting at 1, is 1). This gives us ’return 1;’. How to divide the problem into smaller pieces: The induction step of the proof does exactly this in order to apply the induction hypothesis. We have the following: n X i=1

i=

n−1 X

!

i +n

i=1

The first term of the right hand side is small enough to apply the induction hypothesis on, i.e., we can use that value for a recursive call. Remember that the idea of calling the function on a smaller problem reflects the idea of using the induction hypothesis on smaller problem: we assume that if we pass the recursive function a smaller problem then it will calculate and return the correct value. The second term of the right hand side is what we have do deal with locally in the function. Local computation: As we can see, we do not have to actually do anything with the value n – just combine it with the result of the recursive call to obtain a final result. So the local computation is nothing more than ’n’. Recursive call: The break down of the problem has given us the value of the P parameters to be passed to the recursive call (the value on top of the -sign: n − 1). This means the recursive call looks like ’s (n − 1)’. Combine local computation and result from recursive call: by inspecting the induction proof you will see that the result from using the induction hypothesis and the value n is combined by adding them together – you will have to do the exact same thing in your C++ program. This means that your combine function is simply ’+’. This all combines to yield ’return s(n − 1) + n’. Thus, the finished function looks like this: int s (int n) { if (n == 1) return 1; else return s (n − 1) + n; } 4

However, a faster non recursive version of the same program would be: int s (int n) { return (n ∗ (n + 1))/2; }

3

A string recursion example

In the following example bi is a character ’0’ or ’1’, and bi is defined as: (

bi =

0 : if bi = ’0’ 1 : if bi = ’1’

For a given binary string s = bn−1 bn−2 · · · b2 b1 b0 , bi ∈ {’0’,’1’} of length n, we know that the decimal value can be computed as follows: dec. value of s = bn−1 2n−1 + bn−2 2n−2 + · · · + b2 22 + b1 21 + b0 20 =

n−1 X

2i bi

i=0

How can we construct a recursive function to calculate this value? To devise this algorithm let us assume that we have a function bin2dec that takes in a string (valid bit string) and returns the binary string’s value in decimal. Assuming existence of the function we are writing is the trick that makes it all work (it is the same as assuming the induction hypothesis). This means that bin2dec(“1101“) = 13. How can this “1101“ string be broken into smaller pieces such that we can use the bin2dec function recursively in its own implementation? We know that the value of the least significant bit contributes to the entire sum with either 0 or 1 depending on its value. In the example here the least significant bit is the right most 1. What if we computed bin2dec(“110“), i.e., we stripped the least most significant bit off and called recursively? bin2dec(“110“) = 6, but how does that help figuring out the decimal value of “1101“? Remember that bin2dec(“1101“) = 1 ∗ 23 + 1 ∗ 22 + 0 ∗ 21 + 1 ∗ 20 , and by rewriting this we get: 1 ∗ 23 + 1 ∗ 22 + 0 ∗ 21 + 1 ∗ 20 = (1 ∗ 22 + 1 ∗ 21 + 0 ∗ 20 ) ∗ 2 + 1 ∗ 20

5

and 1 ∗ 22 + 1 ∗ 21 + 0 ∗ 20 = 6 = bin2dec(“110“). So (1 ∗ 22 + 1 ∗ 21 + 0 ∗ 20 ) ∗ 2 + 1 ∗ 20 = bin2dec(“110“) ∗ 2 + 1 ∗ 20 . So by stripping off the least significant bit, calling recursively on the left over string and multiplying the result of the recursive call by 2 and adding the value of the stripped off least significant bit, we have a recursive formula. All we need is a base case, and that can be the string that contains just one bit, so either “0“ or “1“. Thus, we get: bin2dec(b0 ) = b0 bin2dec(bn−1 bn−2 · · · b1 b0 ) = bin2dec(bn−1 bn−2 · · · b1 ) ∗ 2 + b0 which easily can be transformed into the following algorithm: int bitValue(char b) { if (b == 0 10 ) return 1; else return 0; } int bin2dec(int s) { int size = s.size(); if (size == 1) // Base Case Test return bitValue(s[0]); // Base Case Value else // Recursive Case return bin2dec(s.substr(0,size − 1))*2 + bitValue(s[size − 1]); } Can we now use this implementation and the previous specification to prove that function bin2dec actually works? P i We wish to prove that bin2dec(bn−1 bn−2 . . . b1 b0 ) = n−1 i=0 2 bi Of course we will need an induction proof, so we start with the base case: Base case: We can assume that we only work on non empty valid strings. So the basis step is checking strings of length 1. 6

Assume s = b0 . If the size of a string is 1, i.e., it contains only one bit, then the value of that bit is returned: bin2dec(b0 ) = bitValue(b0 ) = b0 = 20 b0 =

0 X

2i bi

i=0

so the base case holds. We now need to formulate an induction hypothesis. Induction hypothesis: Assume ∀ s : | s | ≤ n − 1 that |s | −1

bin2dec(s) =

X

2i bi

i=0

That is, the formula holds for all string s where the length of the string (| s |) is less than n. Written differently, but with the exact same meaning (for a string of length n − 1): bin2dec(bn−2 bn−3 · · · b1 b0 ) = 2n−2 ∗ bn−2 + 2n−3 ∗ bn−3 + · · · + 21 ∗ b1 + 20 ∗ b0 =

n−2 X

2i bi

i=0

for s = bn−2 bn−3 · · · b1 b0 . Induction step: Assume s = bn−1 bn−2 · · · b1 b0 . This is a string of length n. By chopping up this string in to 2 parts; one of length n − 1 and one of length 1 we can use the induction hypothesis on both and then combine the result: bin2dec(bn−1 bn−2 · · · b1 b0 ) = = = = =

bin2dec(bn−1 bn−2 · · · b1 ) ∗ 2 + bin2dec(b0 ) bin2dec(bn−1 bn−2 · · · b1 ) ∗ 2 + 20 b0 (2n−2 bn−1 + 2n−3 bn−2 · · · + 20 b1 ) ∗ 2 + 20 b0 2n−1 bn−1 + 2n−2 bn−2 · · · + 21 b1 + 20 b0 n−1 X

2i bi

i=0

which is exactly what we wanted to show. We used the induction step in the second and the third line, then multiplied through with 2 and obtained the correct result. Also keep in mind that bin2dec(bi ) = bitValue(bi ) = bi . 7

4

An easy one

Assume you have the following program: int fac (int n) { if (n == 0) return 1; else return fac (n − 1) ∗ n; } Prove by induction that fac (n) = n! = n ∗ (n − 1) ∗ · · · 2 ∗ 1. Base case: fac (0) = 1 = 0 ! Induction hypothesis: Assume ∀ k : 0 ≤ k ≤ n − 1 that fac (k) = k!. Induction Step: fac (n) = fac (n − 1) ∗ n = (n − 1) ! ∗ n = n!

5

Recursive case from program. Induction hypothesis.

Palindromes

Palindromes are words (or sentences) that are the same when read backwards. A simple example is the word “madam” or a more interesting one is “saippuakuppinippukauppias”, which just happens to mean salesman of soap dishes in Finnish! How can we design a predicate (a function that always returns either true or false) that accepts as input a string and determines if the string is a palindrome or not? All strings of length 0 or 1 are always palindromes! This serves as a good base case. For the recursive case we can look at the following: s.substr(1,s.size()−2)

s = |{z} x0x1 x2 · · · xn−2xn−1 z

}|

{

| {z }

s[0]

s[s.size()−1]

8

If the length of the string s is greater than 1, then we can compare the first and the last character of the string, i.e., characters at position 0 and n − 1. If these are the same then call recursively on the string in the middle (s.substr(1, s.size() − 2)), combine the result of the recursive call and the local comparison using the && operator. We get the following recursive equation for the pal (string s) predicate: (

pal(s) =

true : n≤1 (s[0] == s[n − 1]) && pal(s.substr(1, n − 2)) : otherwise

Where n = s.size(). It is straight forward to implement this function in C++: bool pal (string s) { int n = s.size(); if (n ≤ 1) return true; else return (s[0] == s[n − 1]) && pal (s.substr(1, n − 2)); } Let us now also prove that this function pal actually does was it is supposed to do. Base case: If | s | ≤ 1 then the length of string s is either 0 or 1, and all string of that length are palindromes. Induction hypothesis: Assume ∀ s : | s | ≤ n − 1 that (

pal(s) =

true : if s is a palindrome. false : if s is not a palindrome.

that is, we assume that the function works for all strings of length less than n. Induction step: let | s | = n. The induction hypothesis gives us that pal(s.substr(1, n − 2)) returns true if s.substr(1, n − 2) is a palindrome, and false if it is not. If the recursive call returned true then the entire string s is a palindrome if s[0] = s[n − 1], and not a palindrome if the recursive call returned false or 9

if s[0] ! = s[n − 1]. This is captured by using the logical && operator on the two terms.

pal(s.substr(1, n − 2) == true pal(s.substr(1, n − 2) == false

s[0] == s[n − 1] s[0] ! = s[n − 1] true false false false

Truth table for pal(s.substr(1, n − 2)) && (s[0] == s[n − 1]) Here is an example of a ’trace’ of a call to pal: 1. call: 2. call: 3. call:

pal (“madam”) = (’m’ == ’m’) && pal (“ada”) ← (Recursive call) pal (“ada”) = (’a’ == ’a’) && pal (“d”) ← (Recursive call) pal (“d”) = true ← (Base case)

Returning: 2. call: 1. call:

6

pal (“ada”) = (’a’ == ’a’) && true pal (“madam”) = (’m’ == ’m’) && true = true

Power

Is it easy to see that the following function p computes xn (prove it!): int p (int x, int n) { if (n == 0) return 1; else return x ∗ p (x, n − 1); } However it is not a very efficient algorithm. I propose the following new function p1 : p1 (x, n) =

  

1 : n = 0. x ∗ p1 (x, n − 1) : if n is odd.   p1 (x2 , n/2) : if n is even. 10

Let us prove that p1 (x, n) = xn , and implement the algorithm. Base case: p1 (x, 0) = 1 = x0 . Induction hypothesis: Assume ∀ k : 0 ≤ k ≤ n − 1 that p1 (x, k) = xk . Induction step: Show p1 (x, n) = xn for both cases, i.e., when n is even and when n is odd. Assume n is odd: p1 (x, n) = x ∗ p1 (x, n − 1) = x ∗ xn−1 = xn

Definition Induction hypothesis

Assume n is even: p1 (x, n) = = = =

p1 (x2 , n/2) n (x2 ) 2 n x2∗ 2 xn

Definition Induction hypothesis

The implementation looks like this: int p1 (int x, int n) { if (n == 0) return 1; else if (n% 2 == 0) return p1 (x ∗ x, n/2); else return x ∗ p1 (x, n − 1); } To make things more interesting let us look at a function p2 defined in the following way:   

p : if n = 0. 2 : if n is even. p2 (x, n, p) = p2 (x , n/2, p)   2 p2 (x , (n − 1)/2, x ∗ p) : if n is odd. 11

Convince yourself that p2 (x, n, 1) = xn , and implement a C++ function to compute p2 . The first part is done most easily by an induction proof. Let us first show that p2 (x, n, p) = p ∗ xn . Base case: p2 (x, 0, p) = p = p ∗ 1 = p ∗ x0 . Induction hypothesis: Assume ∀ k : 0 ≤ k ≤ n − 1 that p2 (x, k, p) = p ∗ xk . Induction step: We must show that p2 (x, n, p) = p ∗ xn . There are two cases: Assume n is even: p2 (x, n, p) = p2 (x2 , n/2, p) n = p ∗ (x2 ) 2 = p ∗ xn

Definition of p2 . Induction hypothesis.

Assume n is odd: p2 (x, n, p) = p2 (x2 , (n − 1/2), p ∗ x) 2

n−1 2

= (p ∗ x) ∗ (x ) = (p ∗ x) ∗ xn−1 = p ∗ xn

Definition of p2 . Induction hypothesis.

This proves that p2 (x, n, p) = p ∗ xn . Now if p = 1 the result follows trivially, that is, we can compute xn by calling p2 with p2 (x, n, 1). Again, implementing this function is straight forward: int p2 (int x, int n, int p) { if (n == 0) return p; else if (n % 2 == 0) return p2 (x ∗ x, n/2, p); else return p2 (x∗x, (n−1)/2, x∗p); }

12

7

Sets and sets of sets – simply power sets

For this example assume we have a Set class available with the following operations: Constructors: Set Set(Set s) – constructs a new set as a copy of s. Set Set() – constructs a new set, initially empty. Methods: Element getElement() – removes an element from a set and returns it (Elements can be anything, even sets). int size() – returns the number of elements in a set. void addElement(Element e) – adds element e to the set. Let us implement a function P that takes as a parameter a set A and returns the power set - which is the set of all subsets of A. Let us first prove the following formula as it helps construct the algorithm. | P(A) |= 2|A| that is, the size of the power set of A is 2 to the power of the size of A. Base case: A = ∅ ⇒| A | = 0. | P(∅) | = | {∅} | = 1 = 20 = 2|A| . Induction hypothesis: Assume ∀ A : 0 ≤ | A | ≤ n − 1 that | P(A) |= 2|A| . Induction step: We must show | P(A) |= 2|A| for | A |= n. Assume A = {a1 , a2 , . . . , an }. | P(A) | = = = = = = =

| P({a1 , a2 , . . . , an }) | | P({a1 , a2 , . . . , an−1 }) ∪ {s ∪ {an } : s ∈ P({a1 , a2 , . . . , an−1 })} | P({a1 , a2 , . . . , an−1 }) | + | {s ∪ {an } : s ∈ P({a1 , a2 , . . . , an−1 })} | 2n−1 + 2n−1 2 ∗ 2n−1 2n 2|A| 13

To be able to apply the induction hypothesis is to remove one element, an from the set A. This makes the set size less than n. This means we can apply the induction hypothesis to the remaining elements in the set. The trick to constructing the power set of A goes like this: if A = ∅ the the power set P(A) = {∅}. if A = {a1 , a2 , . . . , an } then the power set of A can be constructed recursively in the following way: Construct the power set P(A0 ) of A0 = {a1 , a2 , . . . , an−1 }. The power set P(A) consists of all the elements in P(A0 ) plus all the elements in P(A0 ) with the element an added: P(A) = P({a1 , a2 , . . . , an−1 }) ∪ {s ∪ {an } : s ∈ P({a1 , a2 , . . . , an−1 })} The algorithm can be constructed using these facts: Set P (Set A) { if (A.size() == 0) { Set newSet = new Set(); newSet.addElement(new Set()); return newSet; } else { Element e = A.getElement(); // Removes e from A Set pset = P (A); // e ∈ / A, | A | = n − 1. Set result = new Set(pset); // Create a copy of pset while (pset.size() 6= 0) { Element s = pset.getElement(); // Get element from pset. s.addElement(e); // Add e to it result.addElement(s); // Add it to the result } return result; } }

14

8

2 Rabbits, 3 Rabbits, 5 Rabbits ....

The Fibonacci numbers is a well known series of numbers: 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 ... The next number in the series is defined as the sum of the previous 2. The na¨ıve Fibonacci algorithm looks like this: int Fib (int n) { if (n == 0 || n == 1) return 1; else return Fib (n − 1) + Fib (n − 2); } As you will have seen the number of calls to the Fib function to compute Fib (n) is proportional to the sum of the first n Fibonacci numbers, that is, a rather large number, which makes this algorithm inefficient. I have another, more efficient, but also much more complicated algorithm. First a number of definitions. Let "

K=

k11 k12 k21 k22 "

R=

#

r11 r12 r21 r22

"

=

1 1 1 0

#

#

= K n−1

f (n) = r11 + r12 Show: ∀ n ≥ 1 : f (n) = F ib (n). Let us first show the following: " n

∀n≥1:K =

Fib (n) Fib (n − 1) Fib (n − 1) Fib (n) − Fib (n − 1)

Base case: " 1

K =

1 1 1 0

#

"

=

Fib (1) Fib (0) Fib (0) Fib (1) − Fib (0) 15

#

#

Induction hypothesis: Assume ∀ k : 1 ≤ k ≤ n − 1 that "

Kk =

Fib (k) Fib (k − 1) Fib (k − 1) Fib (k) − Fib (k − 1)

#

Induction step: Remember that two matrices are multiplied like this: "

a11 a12 a21 a22

#"

b11 b12 b21 b22

#

"

=

a11 b11 + a12 b21 a11 b12 + a12 b22 a21 b11 + a22 b21 a21 b12 + a22 b22

#

so we get n−1 Kn = K K " #" # Fib (n − 1) Fib (n − 2) 1 1 = Fib (n − 2) Fib (n − 1) − Fib (n − 2) 1 0

"

Fib (n − 1) + Fib (n − 2) Fib (n − 1) Fib (n − 2) + Fib (n − 1) − Fib (n − 2) Fib (n − 2)

"

Fib (n) Fib (n − 1) Fib (n − 1) Fib (n) − Fib (n − 1)

= =

#

#

Now let us show that f (n) = (K n−1 )11 + (K n−1 )12 = Fib (n). "

For n = 1 : K n−1 = K 0 = I =

1 0 0 1

#

: I11 + I12 = 1 + 0 = 1 = Fib (1)

For n > 1 : (K n−1 )11 + (K n−1 )12 = Fib (n − 1) + Fib (n − 2) = F ib (n) We now implement this algorithm according to the following abstract description: int f (int n, Matrix K) { if (n == 1) return K11 + K12 ; else " # 1 1 return f (n − 1, K ); 1 0 }

16

To compute the the n’th Fibonacci number we call "

#

1 0 ) 0 1

f (n,

This implementation is a bit different as the parameter K is used to build up the result in, but it should be fairly easy to see that it works. The previous version used a Matrix data type, which does not exist. The following program implements the same program using 4 integers. int f (int n, int k11 , int k12 , int k21 , int k22 ) { if (n == 1) return k11 + k12 ; else return f (n − 1, k11 + k12 , k11 , k21 + k22 , k21 ); } This function we call like f (n, 1, 0, 0, 1). Of course there is an easier algorithm, but that is not the point, now is it.

9

Binomial coefficients -



n m

 

n The binomial coefficient m , read ’n choose m’, is defined as the number of ways you can choose a subset of m elements from a set with n elements (m < n). The mathematical definition is

!

n n! = m (n − m)! m! Now take a look at the well known Pascal’s triangle: 1 1 1 1 1 1 1

3 4

5 6

1 2

1 3

6 10

15

10 20

17

1 4

1 5

15

1 6

1

Or identified by the use of the

  n m

operator with values for m and n.   0

  0   1

1

  0   1   2

2

2

  0   1   2   3

3

3

3

  0   1   2   3   4

4

4

4

4

  0   1   2   3   4   5 0

  6 0

5

5

5

5

5

  1   2   3   4   5   6 1

6 2

6 3

6 4

6 5

6 6

It seems that the following equality might hold if n > m > 0 : !

!

!

n n−1 n−1 = + m m−1 m for n = m or m = 0 we have !

!

n n = =1 n 0 This is most easily proven by a direct proof: !

!

n−1 n−1 + m−1 m

= = = = = =

(n − 1)! (n − 1)! + ((n − 1) − (m − 1))! (m − 1)! ((n − 1) − m)! m! (n − m)(n − 1)! m(n − 1)! + m(n − m)! (m − 1)! (n − m)(n − m − 1)! m! m(n − 1)! (n − m)(n − 1)! + (n − m)! m! (n − m)! m! (m + n − m)(n − 1)! (n − m)! m! n! (n − m)! m! n m

!

18

Writing the function to calculate this value is not hard if we follow this formula: int choose (int n, int m) { if (n == 0 || n == m) return 1; else return choose (n − 1, m − 1) + choose (n − 1, m); }

10

Just one for the road

Prove n X i=0

i2 =

n(n + 1)(2n + 1) 6

by induction and write a recursive C++ function that implements this function. Oops! Out of space — you finish it!

19

A Last Remark It it worth noting that the rigid structure of one recursive call and one local computation does not always hold. We have seen a number of examples where this model holds 100%. Let me briefly give examples of modified models that will come in handy in your work with recursive functions. • One recursive call, one local computation and one combine function: An example of this is the p (x, n) = xn . Recursive call: p (x, n − 1). Local computation: x. Combine function: +. • Two recursive calls, no local computation and one combine function: An example of this is the Fib (n). Recursive calls: Fib (n − 1) and Fib (n − 2). Local computation: none. Combine function: +. • One recursive call, no local computation and no combine function: An example of this is the following gcd function (m > n): (

gcd (m, n) =

n : if n divides m. gcd (n, m % n) : Otherwise.

Recursive call: gcd (n % m). Local computation: none. Combine function: none. int gcd (int m, int n) { if (m % n == 0) return n; else return gcd (n, m % n); }

20