Error-Detecting and Error-Correcting Codes

Error-Detecting and Error-Correcting Codes

How does a computer store an image?

Error-Detecting and Error-Correcting Codes

An image is an array of colored blocks called pixels. Each color is identified by a binary number.

Error-Detecting and Error-Correcting Codes

color red red-orange orange orange-yellow yellow yellow-green green green-glue blue blue-violet violet black bray white brown beige

decimal 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

binary 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111

In this simplified example, there are 16 different colors. Your computer has 256 (or more) different colors.

Error-Detecting and Error-Correcting Codes

An image with m rows and n columns of pixels is stored as an m ⇥ n matrix of binary numbers.

Image: www.datagenetics.com

Error-Detecting and Error-Correcting Codes

Suppose you want to send a photo to a friend. How does your laptop / phone / tablet do it?

Error-Detecting and Error-Correcting Codes

friend’s&device&

your&device&

Basic idea:

image&

matrix&of&& binary&numbers&

receive&matrix&

encode&numbers&so& errors&in&transmission& can&be&detected&

correct&errors&in& encoded&numbers&

reconstruct& image&

Error-Detecting and Error-Correcting Codes

send&matrix&

An error-correcting code is a method for adding extra 0s and 1s to a binary number message so that errors in transmission can be detected and corrected.

Why is error-correction necessary?

Error-Detecting and Error-Correcting Codes

Examples of errors in transmission:

original message (color) 0010 (orange) 0000 (red)

received message (color) 0110 (green) 1000 (blue)

Error-Detecting and Error-Correcting Codes

Goal: Detect and correct errors in the transmission of binary numbers.

Error-Detecting and Error-Correcting Codes

Z2 = {0, 1} with addition defined as follows: 0+0=0 0+1=1 1+0=1 1+1=0 This is addition modulo 2.

Error-Detecting and Error-Correcting Codes

Zn2 is the set of all vectors with n entries in Z2 .

Error-Detecting and Error-Correcting Codes

Exercise: List all vectors in Z42 .

Hint: Think of such a vector as representing a binary number between 0000 (zero) and 1111 (fifteen).

The vectors in Z42 can represent sixteen different pixel colors.

Error-Detecting and Error-Correcting Codes

Claim: The set Z42 is a vector space over Z2 . (This means the only scalars are 0 and 1 and addition is modulo 2.)

Proof: We can verify each of the ten vector space axioms.

Error-Detecting and Error-Correcting Codes

The set Zn2 is a vector space over Z2 for any positive integer n.

Error-Detecting and Error-Correcting Codes

2

3 0 0 0 1 1 1 1 Let H = 4 0 1 1 0 0 1 1 5. 1 0 1 0 1 0 1

Note: 2

3 0 The columns in H are all the vectors in Z32 except 4 0 5. 0

Error-Detecting and Error-Correcting Codes

To create an error-correcting code for 16 different pixel color messages, we need the null space of the matrix H.

Exercise: Find a basis for Nul(H).

Error-Detecting and Error-Correcting Codes

A basis for Nul(H) is given by the row vectors in the matrix B.

2

1 6 1 B=6 4 0 1

1 0 1 1

1 0 0 0

0 1 1 1

0 1 0 0

0 0 1 0

3 0 0 7 7 0 5 1

Error-Detecting and Error-Correcting Codes

We get a more useful basis for Nul(H) by using the row vectors in the reduced echelon form of B.

Exercise: Compute the reduced echelon form of the matrix B. (Don’t forget to do all computations modulo 2!)

Error-Detecting and Error-Correcting Codes

U is the reduced echelon form of the matrix B.

2

1 6 0 U=6 4 0 0

0 1 0 0

0 0 1 0

0 1 0 1

0 1 1 1

1 0 1 1

3 1 1 7 7 0 5 1

Error-Detecting and Error-Correcting Codes

Since B is row equivalent to U, and U is in echelon form, we know the nonzero rows of U form a basis for the row space of B. Since a basis for the null space of H is given by the row vectors in B, the null space of H is equal to the row space of B. Thus the nonzero row vectors in U form a basis for the null space of H.

Error-Detecting and Error-Correcting Codes

Let A be the matrix U T . Then the columns of A form a basis for the null space of H. 2

6 6 6 6 A=6 6 6 6 4

1 0 0 0 0 1 1

0 1 0 0 1 0 1

0 0 1 0 1 1 0

0 0 0 1 1 1 1

3 7 7 7 7 7 7 7 7 5

A is called the encoding matrix. To encode a binary message (color) v in Z42 , we compute the matrix-vector product Av.

Error-Detecting and Error-Correcting Codes

Example: 2

3 1 6 1 7 7 message: v = 6 4 0 5 1

2

6 6 6 6 coded message: Av = 6 6 6 6 4

1 1 0 1 0 0 1

3 7 7 7 7 7 7 7 7 5

The coded message is always equal to the original message with extra 0s and 1s added to the end. These extra digits will be used to detect and correct errors in transmission. Error-Detecting and Error-Correcting Codes

2

3 1 6 1 7 7 Instead of the message (color) 6 4 0 5, your device sends the 1 2 3 1 6 1 7 6 7 6 0 7 6 7 7 code 6 6 1 7 (color plus error detecting / correcting digits). 6 0 7 6 7 4 0 5 1

Error-Detecting and Error-Correcting Codes

This system for creating 7-digit codes by appending three digits to 4-digit messages (colors) is called the Hamming (7, 4) code.

How do we detect and correct errors in Hamming codes?

Error-Detecting and Error-Correcting Codes

Coded messages are vectors in Z72 . Let e1 , e2 , . . . , e7 denote the standard basis vectors for Z72 . 2

6 6 6 6 e1 = 6 6 6 6 4

1 0 0 0 0 0 0

3

2

7 6 7 6 7 6 7 6 7 , e2 = 6 7 6 7 6 7 6 5 4

0 1 0 0 0 0 0

3

2

7 6 7 6 7 6 7 6 7 , . . . , e7 = 6 7 6 7 6 7 6 5 4

0 0 0 0 0 0 1

3 7 7 7 7 7 7 7 7 5

Error-Detecting and Error-Correcting Codes

Let x denote an encoded message in Z72 .

Adding one of the vectors e1 , e2 , . . . , e7 to x is equivalent to making an error the transmission of x.

Error-Detecting and Error-Correcting Codes

Example:

encoded message:

message with error:

2

2

6 6 6 6 6 6 6 6 4

0 1 1 0 0 1 1

3

7 7 7 7 7=x 7 7 7 5

6 6 6 6 6 6 6 6 4

0 0 1 0 0 1 1

3

7 7 7 7 7 = x + e2 7 7 7 5

Error-Detecting and Error-Correcting Codes

Since x is an encoded vector, we have 2 3 v1 6 v2 7 7 x = Av = [a1 a2 a3 a4 ] 6 4 v3 5 = v1 a1 + v2 a2 + v3 a3 + v4 a4 . v4

This means x is a linear combination of the columns of A, which are the basis vectors for Nul(H). This means x is in Nul(H).

Error-Detecting and Error-Correcting Codes

Theorem 1: If x is in Nul(H), then x + ei is not in Nul(H) for i = 1, 2, . . . , 7.

2

3 0 0 0 1 1 1 1 H = 4 0 1 1 0 0 1 1 5. 1 0 1 0 1 0 1

Error-Detecting and Error-Correcting Codes

Proof: Let h1 , h2 , . . . , h7 denote the columns of H. If x is in Nul(H), then Hx = 0. Also Hei = 0 · h1 + · · · + 1 · hi + · · · + 0 · h7 = hi . Since H(x + ei ) = Hx + Hei = 0 + hi = hi 6= 0, we can conclude that x + ei is not in Nul(H).

Error-Detecting and Error-Correcting Codes

Theorem 1 tells us how to detect errors in Hamming codes.

We know that all encoded messages x are in Nul(H), and that messages with an error have the form x + ei . The theorem tells us that messages with an error are not in Nul(H).

This means that we can detect an error by computing Hx. If the product is 0, then there is no error. If the product is not 0, then there is an error.

Error-Detecting and Error-Correcting Codes

Theorem 2: If Hx = hi , then x + ei is in Nul(H), and x + ej is not in Nul(H) for j 6= i. Proof: If Hx = hi , then H(x + ei ) = Hx + Hei = hi + hi = 0. If j 6= i, then H(x + ej ) = Hx + Hej = hi + hj 6= 0.

Error-Detecting and Error-Correcting Codes

Theorem 2 tells us how to correct errors in Hamming codes.

If Hx = hi , then an error has been detected (since hi 6= 0). This error must be in position i because changing the entry in position i results in the vector x + ei , and this vector is in Nul(H), while changing any other entry results in the vector x + ej , which is not in Nul(H). The vector x + ei is the corrected code.

Error-Detecting and Error-Correcting Codes

Examples: 2

6 6 6 6 The received code is x = 6 6 6 6 4

0 1 0 0 1 0 1 2

6 6 6 6 Hx = [h1 h2 h3 h4 h5 h6 h7 ] 6 6 6 6 4

3

7 7 7 7 7. Is there an error? 7 7 7 5 0 1 0 0 1 0 1

3

7 7 7 7 7 = h2 + h5 + h7 7 7 7 5 Error-Detecting and Error-Correcting Codes

2

3 0 0 0 1 1 1 1 H=4 0 1 1 0 0 1 1 5 1 0 1 0 1 0 1 2

3 2 3 2 3 2 3 0 1 1 0 Hx = h2 + h5 + h7 = 4 1 5 + 4 0 5 + 4 1 5 = 4 0 5 0 1 1 0 There is no error.

Error-Detecting and Error-Correcting Codes

2

6 6 6 6 The received code is x = 6 6 6 6 4

0 1 1 1 0 0 1 2

6 6 6 6 Hx = [h1 h2 h3 h4 h5 h6 h7 ] 6 6 6 6 4

3

7 7 7 7 7. Is there an error? 7 7 7 5 0 1 1 1 0 0 1

3

7 7 7 7 7 = h2 + h3 + h4 + h7 7 7 7 5 Error-Detecting and Error-Correcting Codes

2

3 0 0 0 1 1 1 1 H=4 0 1 1 0 0 1 1 5 1 0 1 0 1 0 1 2

3 2 3 2 3 2 3 2 3 0 0 1 1 0 h2 + h3 + h4 + h7 = 4 1 5 + 4 1 5 + 4 0 5 + 4 1 5 = 4 1 5 0 1 0 1 0 Since Hx = h2 , there is an error in position 2.

Error-Detecting and Error-Correcting Codes

2

6 6 6 6 The received code is x = 6 6 6 6 4

0 1 1 1 0 0 1

3

7 7 7 7 7. 7 7 7 5

2

6 6 6 6 The corrected code is x + e2 = 6 6 6 6 4

0 0 1 1 0 0 1

3

7 7 7 7 7. 7 7 7 5

Error-Detecting and Error-Correcting Codes

Error-Detecting and Error-Correcting Codes