Error Correcting Codes (SS14) Carsten Damm (Institut für Informatik, Universität Göttingen) ]Sage-notebook:
1
Contents 1
2
3
Introduction
1
1.1
Basic notions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2
Minimum distance decoding (the combinatorial perspective) . . .
1.3
An interesting code (the algebraic perspective)
1.4
Decoding rules (the statistical perspective)
. . . . . . . . . . . .
13
1.5
Coding and complexity (the algorithmic perspective) . . . . . . .
17
1.6
Exercises
18
. . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Some code constructions
1 6 10
19
2.1
Check digit systems and code extension
. . . . . . . . . . . . . .
2.2
Code equivalence . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
2.3
Hadamard codes and their decoding
24
2.4
Short Detour: Local decoding and local testing
2.5
Polynomial codes and Berlekamp-Welch-algorithm
2.6
Exercises
. . . . . . . . . . . . . . . . . . . . . . . . . .
19
30
. . . . . . . .
33
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
40
Linear Codes
41
3.1
De nition and rst properties . . . . . . . . . . . . . . . . . . . .
41
3.2
Generator matrix and encoding . . . . . . . . . . . . . . . . . . .
42
3.3
Parity check matrix
. . . . . . . . . . . . . . . . . . . . . . . . .
44
3.4
Syndrome decoding . . . . . . . . . . . . . . . . . . . . . . . . . .
46
3.5
Application to certain Reed-Solomon codes
. . . . . . . . . . . .
49
3.6
The dual of a code and MacWilliams identity . . . . . . . . . . .
53
3.7
Majority decoding for linear codes
. . . . . . . . . . . . . . . . .
56
3.8
LDPC codes and expander graphs
. . . . . . . . . . . . . . . . .
60
3.9
Complexity of decoding
. . . . . . . . . . . . . . . . . . . . . . .
65
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
68
3.10 Exercises
i
4
5
6
7
8
9
Existence and construction of good codes
69
4.1
Combinatorial upper bounds
. . . . . . . . . . . . . . . . . . . .
69
4.2
Combinatorial lower bounds . . . . . . . . . . . . . . . . . . . . .
73
4.3
Asymptotic bounds . . . . . . . . . . . . . . . . . . . . . . . . . .
75
4.4
Good codes
78
4.5
Error rates of good codes
4.6
Random codes
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
83
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
85
Source and channel coding
88
5.1
Uncertainty, entropy and information . . . . . . . . . . . . . . . .
89
5.2
Data compression and source coding theorem
. . . . . . . . . . .
89
5.3
Channel Capacity and channel coding theorem
. . . . . . . . . .
90
Covering codes
92
6.1
q -nary
Hamming codes . . . . . . . . . . . . . . . . . . . . . . . .
92
6.2
Golay codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
94
6.3
Classi cation of perfect codes . . . . . . . . . . . . . . . . . . . .
94
6.4
Covering radius . . . . . . . . . . . . . . . . . . . . . . . . . . . .
95
Cyclic codes
99
7.1
First examples and considerations
. . . . . . . . . . . . . . . . . 100
7.2
Generator polynomial and generator matrix . . . . . . . . . . . . 102
7.3
Parity check polynomial and syndrome polynomial
7.4
Cyclic Hamming codes . . . . . . . . . . . . . . . . . . . . . . . . 119
7.5
BCH bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
7.6
Decoding BCH codes . . . . . . . . . . . . . . . . . . . . . . . . . 119
7.7
Exercises
7.8
Examples and rst properties . . . . . . . . . . . . . . . . . . . . 119
. . . . . . . . 109
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Reed Solomon codes
119
8.1
Classic and historic view . . . . . . . . . . . . . . . . . . . . . . . 119
8.2
Properties of MDS codes . . . . . . . . . . . . . . . . . . . . . . . 119
8.3
Decoding RS-codes . . . . . . . . . . . . . . . . . . . . . . . . . . 119
8.4
PGZ decoder
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
8.5
Goppa codes
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
8.6
Berlekamp Massey decoder
8.7
Sugiyama decoder
8.8
Application: QR codes . . . . . . . . . . . . . . . . . . . . . . . . 119
8.9
Exercises
. . . . . . . . . . . . . . . . . . . . . 119
. . . . . . . . . . . . . . . . . . . . . . . . . . 119
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Building new codes from old ones, Part II
(u, u + v)-construction
119
9.1
The
. . . . . . . . . . . . . . . . . . . . . 119
9.2
Concatenation and burst errors . . . . . . . . . . . . . . . . . . . 119
9.3
Zyablov's trade-o and Justesen codes . . . . . . . . . . . . . . . 119
9.4
Interleaved codes . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
9.5
Application: Data storage on CD and DVD . . . . . . . . . . . . 119
ii
Preliminary Remarks This le is mainly intended for my own book keeping and perhaps occasional use during the lecture.
In particular:
I do not promise to keep these notes
consistent and to continuously polish and publish them. Further, the le is not intended for redistribution. It is mainly based on my German lecture notes on the subject This le contains lots of internet links which might be invisible in printed versions. For better reference small portions in this document (paragraphs) are numbered within sections (printed in the right margin of each page).
Added in proof [2014-04-10 Thu] This course heavily relies on the computer algebra system Sage and one of the intended outcomes of this course are skills in making eective use of Sage. So participants are required to program in Sage, produce corresponding worksheets and present them in exercise sessions. Here are some hints on how to start programming Sage:
http://www.citeulike.org/user/damm/tag/sage
(will be completed from
time to time). Currently this text is (almost) the same as I left it in summer 2012.
In par-
ticular, embedded links to Sage-worksheets direct to IfI's old Sage server
https://sage2.informatik.uni-goettingen.de
running Sage v.4.8.
Mean-
while a new server was set up, that runs Sage v.6.1.1:
https://sage.informatik.uni-goettingen.de.
As the semester proceeds, I will update and change the worksheets to make them run on the new server.
If time permits, I will also update portions of
text in these notes, maybe also reorder some topics, and ll some gaps. These changes come without warning!
iii
iv
1
Introduction
1.1 Basic notions
The general setup
1-1
Error control coding
1-2
Measures taken to protect messages from channel errors are summarized as error control coding:
backward error correction (BEC)
Bob requests retransmission after
noticing errors Example:
ARQ - automatic repeat request, receiver acknowledges
(ACK) every correctly received package, sender retransmits packages with missed ACKs
forward error correction (FEC)
Alice cleverly encodes messages to
enable recovering from error Example: repeat bits, append check bits, ...
Message, code, and ambient space
1-3
Arecibo message - sent out in 1974 to reach extraterrestrial civilizations
http://www.daviddarling.info/images/AreciboM.jpg
message = sequence of symbols from a nite alphabet
1
Sage-notebook: Code and ambient space
binary alphabet
= a two-symbol alphabet
B
(symbols deliberately denoted 0 and 1 - bits)
binary code = distinguished set of strings binary block code of length
Bn
n
= subset
∈ B∗
(codewords)
n 1
C⊆B
is the ambient space of the code
Channel coding
1-4
channel = device transporting bit sequences may suer from noise (e.g., occasional bit ipping) in particular synchronized: input bits correspond to output bits in same order, no deletes, no inserts
C = {c0 , c1 , ..., c2k −1 } ⊆ Bn
is called binary
(n, k)-code
to minimize eect of errors:
* *
transform messages into codewords before sending
for
expansion) by
k -bit blocks, encode blocks seperately 0 ≤ i < 2k encode s = bink (i) (k bit binary ck
split bit stream into Example:
Coding schemes
1-5
De nition 1.1. A binary
(n, k)
coding scheme is a pair
(E, D)
of map-
pings
E : Bk → Bn , D : Bn → Bk , such that
D
D(E(s)) = s
for each message
s ∈ Bk .
is allowed to be a partial mapping (not de ned for every input).
the set
{C(s)|s
is a message} (= set of codewords) is the correspond-
ing code
C
C, D
is called encoder,
D
are usually given by algorithms
in technical contexts
is called decoder
(C, D)
is called a codec
decoding = correction + message extraction
correction based on received noisy word make a good guess on the codeword, which was presumably sent (estimated codeword)
1 in
sequel: will use code and block code synonymously
2
1-6
message extraction given a codeword, extract an encoded message from the estimated codeword (this amounts to inverting the encoder, i.e., apply
E −1
to the estimated codeword)
Convention
1-7
Remark The interesting part of decoding is correction - inversion turns out to be trivial in most cases. Therefore: Will sometimes use terms correction and decoding inter-
changeably. Examples to follow.
Analogue vs. digital
1-8
technical channels mostly rely on dissemination of continous physical values (like charge)
we assume corresponding analogue/digital and digital/analogue converters (not discussed in this course)
Fully featured communication system
3
1-9
(from lecture notes by John Gill)
Explanation
1-10
A communication system consists of a sequence of blocks, each transforming some input signal into an output signal.
w.l.o.g.2 all transformations are considered error-free, only the channel may suer from noise
to protect signals from channel noise we do error correction coding (ECC) (channel coding) central idea of ECC: add useful redundancy
encryption/decryption serve to ensure privacy/integrity/authenticity and modulation/demodulation adapt to the physical needs of the channel won't be considered in this course
Notation used throughout this course
selected message
s ∈ Bk
encoded message = sent codeword received word
1-11
c ∈ C ⊆ Bn
r ∈ Bn
(single transmission is speci ed by the pair
estimated codeword
c 7→ r)
b c ∈ C ⊆ Bn
estimated message = back transformed estimated codeword
b s ∈ Bk
important parameters:
n
M
k = log M
code length = code size (= # of codewords) (# of message bits)
Transmission rate
2 this
1-12
is non-trivial!
4
each codeword of a binary
(n, k)-code C
is speci ed by
k
message
bits
the quantity
k/n is called R(C))
(transmission or information) rate of the
code (denoted
in general
R(C) = where
log
dlog |C|e , n
denotes the binary logarithm
The length
Examples 1.2. codewords
0n = |{z} 0...0
and
n
n binary repetition code REPn consists of 1n = |{z} 1...1. Its information rate is 1/n n
(low).
The length
n single parity check P code SPCn consists of all c ∈ Bn whose parity ci is even. Its information (n − 1)/n (high). quences
bit serate is
Error and error weight
De nition 1.3. Let
c, r
1-13
be sent/received word of a transmission.
e = (e1 , ..., en ) ∈ Bn ( 1 , if ri 6= ci ei = 0 , else.
error of this transmission is
The
de ned by
The quantity
|e| := |{i : ei 6= 0}| Sage-command(s):
is called error weight
e.hamming_weight()
Noise equation
B
is a group under addition modulo 2 (as well as
Bn )
returns weight vector e 1-14
the noise equation of a transmission:
r=c+e expresses the in uence of errors by arithmetic operations
this equation is understood componentwise and modulo 2 Hamming distance: for
x, y ∈ Bn
let
d(x, y) := |{i : xi 6= yi }|
Observation (trivial but important): The error weight equals the Hamming distance between sent and reveived word:
|e| = d(c, r)
Exercises
1-15
5
of
B consider the alphabet S = {+1, −1}. The ambient space Sn . How does the noise equation for a transmission c 7→ r look like in the case c, r ∈ Sn ?
1. Instead of
in this case is
2. Bit (or sign) ipping errors are called Hamming errors. type of errors is erasures: unrecognizable.
Another
A received bit is erased if its value is
Generalize the notion of a coding scheme to this
situation and try to de ne a suitable version of error equation that expresses error events arithmetically.
Hint Make use of the previous exercise. 3. Convince yourself that Hamming distance satis es the axioms of a
metric, i.e, for all
x, y, z ∈ Bn
d(x, y) = d(y, x)
d(x, y) ≥ 0
d(x, y) + d(y, z) ≥ d(x, z)
holds:
and equality holds only in case
x=y
1.2 Minimum distance decoding (the combinatorial perspective)
A toy example
1-16
Example 1.4. We consider 3 encodings for message space 1. add a parity bit:
{00, 01, 10, 11}.
(s1 , s2 ) 7→ (s1 , s2 , s1 + s2 (mod 2))
single bit errors can be detected, but not corrected, e.g., distance 1 from
000, 110,
2. repeat message 3 times:
and
100
has
101
(s1 , s2 ) 7→ (s1 , s2 , s1 , s2 , s1 , s2 )
single bit errors can be corrected, e.g.,
101110 7→ 101010
3. repeat message 2 times and add a check bit for second repetition
(s1 , s2 ) 7→ (s1 , s2 , s1 , s2 , s1 + s2 (mod 2)) single bit errors can be corrected, e.g.,
10111 7→ 10101
Minimum distance decoding (MDD)
correct reveived word
r∈B
1-17
to closest codeword
closest means: 3 will
d(c, r)
b c∈C
3
- this seems to be the most natural decoding rule takes minimum at
justify it later
6
c=b c
C
is said to correct error
closest to
c+e
e,
if for all codewords
is unique and equals
c ∈ C,
the codeword
c:
arg min d(c + e, c0 ) = {c} c0 ∈C
the closest codeword to
r
is denoted
MDC (r), if it is unique
Correcting all errors up to some weight
Examples 1.5.
REP2t+1
1-18
corrects all errors of weight
≤t
(simply by
setting each bit to the majority value)
SPCn
cannot correct any error!
any codeword
C
is called
t
c
Let, e.g.,
e = (1, 0, ..., 0), then c + e)
for
there are several closest codewords to
error correcting if it corrects all errors of weight
≤t
Minimum distance
1-19
the minimum distance of a code
C
is de ned as
d∗ (C) := min{d(c1 , c2 ) : c1 , c2 ∈ C, c1 6= c2 } a pair of distinct codewords whose distance is
d∗
is called minimal
pair of the code
t∗C := b d
∗
−1 2 c is called error tolerance of the code
On the error tolerance of a code
Theorem 1.6.
Proof.
C
1-20
∗ ∗ is tC error correcting but not tC
by triangle inequality: because of
|e| ≤ t∗C
have several closest codewords (dark blue)
7
+1
error correcting.
no word
c+e
can
it is easy to construct some non-correctable error of weight t∗C
+1
(light blue)
Detecting errors
C
1-21
is said to detect error
e,
if for any
c
the received word
c+e
is not
a codeword
Example: SPCn detects any odd-weight error the error detection radius of a code is given by
max{r|∀c ∈ C : |e| ≤ r ⇒ c + e 6∈ C} From the de nition follows:
Theorem 1.7. The error detection radius of a code is
d∗ − 1.
Complete and incomplete MD decoders
1-22
Remark correcting/detecting are purely combinatorial notions, don't say anything about how to correct or detect by an algorithm
an algorithm, that assigns to each
r ∈ Bn
some closest codeword is
called complete MD decoder
for any code complete MD decoding can be achieved by exhaustive search (but sometimes better algorithms are known)
Example 1.8 (Python-like implementation of exhaustive search).
def MDD(r,L): # L = list of codewords d = n+1 # this is infinity for a code of length n for i in range(1,M): d1 = dist(r,L[i]) if (d1 < d): d = d1 c = L[i] return(c)
an algorithm, that nds closest codewords only for a subset of
Sage-notebook: Exhaustive search
Bn
is called an incomplete minimum distance decoder (example given in following exercises)
Hamming balls and correction radius
the Hamming ball of radius
t
centered at
1-23
x∈B
Bt (x) := {z : d(x, z) ≤ t}. 8
n
is
c ∈ C and t ≤ t∗C MD decoders correct balls Bt∗ (c) are called correction balls C
since for any codeword
r ∈ Bt (c)
to
c,
the
for a general decoder certain codeword
c
D : Bn → C
each
the set of words corrected to a
is called it's decoding region:
BD (c) := {r : D(r) = c}
Decoding regions of common decoding principles
1-24
Screenshot taken from Lecture Notes U Bremen (Bockelmann/Wübben)
Decoding regions of general decoders
the maximum of all
t
such that
correction radius of decoder
Bt (c) ⊆ BD (c)
1-25
for all
c∈C
is called
D
Exercises 1. Joke question:
1-26 How many erasures can be detected by a code of
minimum distance
d?
2. How many erasures can be corrected by a code of minimum distance
d? C = {c1 , ..., cM } ⊆ Bn . Let d∗ = t + s + 1 with s > t. Write a Python-like decoder for C that corrects all errors up to weight t and
3. Let
9
detects all errors up to weight
s.
Determine the correction radius of
your algorithm.
1.3 An interesting code (the algebraic perspective)
Check bits for error correction
REP3
1-27
can correct single bit errors using overhead of 2 check bits
similar: error protect 4 bits using overhead of 8 check bits now: protect 4 bits using only 3 check bits code length is 7
A (7,4) Hamming code
let
H
be a
3×7
1-28
Hamming-matrix:
1 H = 0 1
1 1 0
1 1 1
0 1 1
1 0 0
0 1 0
0 0 1
(property: columns = all binary non-zero columns of length 3)
the codewords are the 7-bit vectors
x = (x1 , ..., x7 )
that satisfy
H · xT = 0T 3 (addition/multiplication modulo 2)
this is the rst example of a linear code
Code size
1-29
codewords = solutions of a linear equation system #variables - rank = dimension of solution space = 4 therefore: code size (= # of codewords) is
M = 24 = 16
the code is called (7,4)-Hamming code and we denote it
HAM3
(3
parity checks = 3 redundant bits)
Pictorial description
1-30
10
Encoding
1-31
simple encoding:
*
place message bits into regions 1, 2, 3, and 4
*
place corresponding parity bits into regions 5, 6, and 7
redundance bits are simply appended to message bits more general: encoding
E : Bk → Bn
is called systematic, if message
bits appear unscrambled in the codeword
Correction capability
Lemma 1.9. Proof.
HAM3
1-32
is 1-error correcting.
ill circle: parity check condition not satis ed
observe: for any ill/healthy circle combination there is at most one bit contained in every ill and in no healthy circle
cure all circles by correcting that single bit, hence every single bit error is corrected
Minimum distance
Corollary 1.10. Proof.
1-33
d∗ (HAM3 ) = 3
by Theorem 1.6 on error tolerance:
1000101 is a codeword, hence
d∗ ≥ 3
d∗ ≤ d(07 , 1000101) = 3 Sage-notebook: (7,4)-Hamming code
Exercises 1. Prove that
1-34
HAM3
can correct any 2 erasures.
2. Can it also correct some 3 erasures? What about 4 erasures?
11
3. Prove that if
c
is a
bitwise complement
HAM3 c
codeword, then the same holds for its
4. Is there a distance 5 codeword pair in
q -nary
HAM3 ?
alphabet
will use
Q
1-35
as shorthand notation for any particular
for code size
M
and length
n
q -nary
alphabet
the information rate is de ned as
R(C) =
logq M n
to hold up noise equation Example:
Q must be a group Zq = ({0, 1, ..., q − 1}, + mod q)
Groups and fields
B
is even a eld:
1-36
B = F2 = ({0, 1}, + mod 2, · mod 2)
advantage: use of linear algebra to de ne codes Example: Hamming code is a speci c linear subspace
will later use (linear) algebra even for encoding and decoding generalizations to q -nary alphabet:
* Fq = ({0, 1, ..., q − 1}, + mod q, · mod q) * Fq
Galois eld of prime power order
for primes
q
q = pn
Important notation If
Q
is a ring, the multiplicative subgroup (= the set of elements that
have a multiplicative inverse) is denoted If
Q
is even a eld, then clearly
Q∗ .
Q∗ = Q \ {0}.
Short aside on finite fields
Zq , q > 1
if
q
1-37
is a group w.r.t. addition
is not a prime,
Zq \ {0} is not
a group under multiplication, since
it has zero divisors Example:
2 · 3 ≡ 0 mod 6
common notation:
Z∗q = {a ∈ Zq |∃b ∈ Zq : a · b ≡ 1 mod q}
unit subgroup
if
q
is prime
Z∗q = Zq \ {0}
12
the
hence:
Fq := (Zq , + mod q, · mod q) is a eld if and only if q
[construction of
Fq
for prime powers
q
is prime
is obviously more involved]
Exercises
1-38
x 7→ (−1)x
1. Observe that
is a group isomorphism
earlier exercise) is a subgroup of real numbers6= 2. Let
m>1
0
and
·
(R∗ , ·)
where
R∗
B→S
and
S
(see
denotes the set of
denotes usual multiplication.
be an integer. Try to nd a similar group
Sm
with usual
multiplication as group operation that is isomorphic to ({0,1,. . . ,m-
modm).
1},+
1.4 Decoding rules (the statistical perspective)
Error correction as a game
1-39
Alice: selects codeword according to some probability dist channel: distorts event
c 6= b c
c 7→ r,
Bob: estimates
r 7→ b c
is called word error, penalty function(each word error
costs 1 Euro):
( 0 p(c, b c) = 1
, if , if
c=b c c 6= b c
risk of false decoding depends on:
*
probability distribution
*
probability
PE
PC
on message space (Alice's choice)
of transmission errors (channel statistics)
Maximum a posteriori estimation (MAP-decoder)
1-40
minimum error rule: minimize risk = maximize probability of correct decoding
let
r
be given ( xed), then the decoder maps
7→ arg max P[c|r]
MAP : r
c∈C
impractical: 1. requires knowledge of
PC
2. hard to apply for large
M
(code size)
Maximum likelihood estimation (ML-decoder)
13
1-41
next best thing: don't maximize probability but likelihood!
MLDC (r)
:=
arg max P[r|c] c∈C
Remark: MAP maximizes
P[model|data].
ML maximizes
P[data|model].
This letter term is called likelihood of the model. There is no o cial German translation to likelihood, but Plausibilität comes close to it.
equivalent to MAP-decoding in case of uniform distribution of codewords:
P[c, r] MAP(r) = arg max P[c|r] = arg max P[r] c∈C c∈C PC [c] = arg max P[r|c] · P[r] c∈C and by uniformity
= arg max (P[r|c]) c∈C
easier to handle: by noise equation likelihood depends only on channel properties, since
P[r|c] = PE [r − c]
Binary symmetric channel (BSC)
any bit has probability
1-42
p < 1/2
of being ipped in the channel
errors occur independent for every bit being transmitted
p
is called crossover probability
by independence: the probability of error transmission is
w
e ∈ Bn
of weight
w
in a
n−w
pw := p (1 − p)
observe:
p0 ≥ p1 ≥ p2 ≥ ... ≥ pn
MD-decoding = ML-decoding for the BSC
Corollary 1.11. For the binary symmetric channel with crossover probability
p < 1/2
holds
MLDC (r) = MDC (r). This is a formal justi cation for MD decoding.
14
1-43
Error rates
1-44
bit error rate (BER)
technical term for crossover probability
word error rate (WER)
probability that a chosen encoded message
will be wrongly decoded
WER depends on
For
*
prob dist of messages
*
prob dist of channel errors
*
coding scheme
(n, k)-codecs
s/codewords c e
E, D
the WER is given by
WER(E, D) = Es∈Qk [Pe∈Qn {D(E(s) + e) 6= s}] .
Word error rate of BSC
1-45
in case of uniform distribution on the message space and binary symmetric channel the most natural codec is based on ML-decoding
corresponding WER is
WER(C)
= =
1 X Pe∈Bn {D(c + e) 6= s} 2k c∈C X 1 X p|e| (1 − p)n−|e| , k 2 c∈C e: D(c+e)6=s
Bounded minimum distance decoding
1-46
di cult to apply:
upper bound: WER of bounded minimum distance decoding (decoding up to error weight
t∗ )
n X n w WER(BMD) = p (1 − p)n−w w w=t∗ +1 Proof: in case of BMD holds
WER = P[
15
error of weight
> t∗ ]
WER of repetition code
if
Example 1.12. if and only if
n odd, |e| > n/2
1-47
e ∈ Bn
then
implies word error for
REPn ,
in this case BMD coincides with BDD:
X n WER(REPn ) = pw (1 − p)n−w . w n 2
0:
there is no degree
< k
polynomial with value
r N (x)
the idea is now, to look instead for a rational function L(x) with (mainly) that value table
will show, that such
N (x), L(x)
exist and how to derive
s(x)
Details
2-44
more precisely, we look for polynomials
N (x), L(x)
(not both zero)
that satisfy the following conditions 1.
∀i : N (bi ) = L(bi ) · ri
this corresponds to the value table
property 2.
L(x) is a monic polynomial (i.e., leading coe cient = 1) of degree t
3.
deg N (x) ≤ t + k − 1
the following pair satis es these conditions
* L(x) = ΛR (x)
given by
ΛR (x) = Πei 6=0 (x − bi )
37
* N (x) = s(x) · ΛR (x),
where
s(x) = s0 + s1 x + ... + sk−1 xk−1
is
the message polynomial moreover
s(x)
can be computed by polynomial division from this
particular solution
s(x) =
N (x) L(x)
any solution to 1.-3. is as good as the above one by the following lemma
Lemma 2.16. If
N (x), L(x)
and
M (x), K(x)
are solutions to 1.-3., then
N (x) M (x) = . L(x) K(x)
Proof
2-45
by 1.: for all
i
holds
N (bi ) = ri · L(bi )
ri · K(bi ) = M (bi )
and
multiply these equations to obtain
∀i : ri N (bi )K(bi ) = ri L(bi )M (bi ) if ri
6= 0
canceling on both sides gives
if ri
=0
we know
N (bi )K(bi ) = L(bi )M (bi )
N (bi )K(bi ) = L(bi )M (bi ) = 0
(by 1.), which gives
∀i : N (bi )K(bi ) = L(bi )M (bi ) thus, the degree
n
1, n = 2r − 1
and
H ∈ Br×n
a binary Hamming matrix. The code
HAMr = ker H is called binary Hamming code of order it the
(n, n − r)
r
(in technical writing people call
Hamming code). It is unique up to equivalence.
Examples 3.9. some check matrices for binary Hamming codes (for
r=3
see example above)
011 r = 2: H = 101 code REP3
,
HAM2
coincides with the length 3 binary repetition
45
Sage-notebook: Binary Hamming codes 3-11
r = 4:
0 0 0 1 HAM4
0 0 1 0
0 0 1 1
0 1 0 0
0 1 0 1
0 1 1 0
0 1 1 1
1 0 0 0
1 0 0 1
1 0 1 0
1 0 1 1
1 1 0 0
1 1 0 1
1 1 1 0
1 1 . 1 1
has length 15 and dimension 11.
Binary Hamming codes: Minimum distance and perfectness
columns of a Hamming matrix
H
3-12
are pairwise dierent, so any two
of them are linear independent (see special case of Main Lemma)
the sum of any two columns of
H
is another column, hence there is
a size 3 linear dependent set of columns
hence:
d∗ (HAMr ) = 3
for any
r≥2
Corollary Binary Hamming codes are perfect (parameters
as
Proof.
d∗ = 3,
[2r − 1, 2r − 1 − r, 3]).
any two radius 1 balls around codewords are dis-
joint
the volume ( = cardinality) of a radius
1 + n = 2r − 1 + 1 = 2r
and there are
hence their total volume is
2n
1 ball in Hamming 2k = 2n−r such balls
they completely ll up
space is
Bn
3.4 Syndrome decoding
Recall: Maximum likelihood decoding
let
C ⊆ Qn
be a code (not necessarily linear)
consider a transmission
e∗
3-13
c∗ ; r = c∗ + e∗
is unknown to receiver Bob
in Bobs situation any vector explain why
r
hence: the set
e
of shape
e = r − c, c ∈ C
would
was received
r − C := {r − c : c ∈ C}
in this transmission
46
is the set of potential errors
by the maximum likehood strategy the receiver selects the minimum weight vector in
r−C
as an estimation for
e∗
ˆ = arg min |r − c| e c∈C
the potential error sets may look dierent for dierent received works, so in general each
Qn /C
r
has to be handled individually
cosets
3-14
if the code
r−C
C = ker H
is linear, several of the potential error sets
coincide
Qn modulo C x ∈ Qn
more precisely: these sets are cosets of shape
x + C := {x + c : c ∈ C},
the coset containing
r
given by
r is yrT := HrT
where
i.e., sets of
in 1-to-1 correspondence to the syndrome of
Sage-command(s): from sage.coding.decoder import syndrome;
proof:
00
0
coset r
00
r −C =r −C ⇔r −r ∈C ⇔ H(r0 − r00 )T = 0 ⇔ Hr0T = Hr00T thus the syndrome is some kind of hash function for identifying cosets
note that some cosets may contain several minimum weight vectors Example: Hill's code:
01010 + CHill = {01010, 10110, 01101, 10001}
there are two vectors of minimum weight in this coset
Coset leaders and syndrome decoding
let r1
+ C, r2 + C, ..., rN + C, N = q r
3-15
be the complete list of cosets
consider a selection of vectors f1 , f2 , ..., fN
∈ Qn ,
such that ∀i : fi is ri + C and the corresponding syndrome T : Qr → Qn , such that T (yT ) = fiT if
a minimum weight vector in
table, i.e., the mapping H · fiT = yT
the chosen vectors are called coset leaders syndrome decoding with this particular syndrome table works as follows:
*
returns as a list
syndrome(C,r)
0
upon receiving
r ∈ Qn
compute the syndrome
47
y
+ C
*
compute the corresponding coset leader
*
return
in short:
fiT = T (yT )
r − fi
ˆ = r − T (H · rT )T c
Consequence An error if
e
e
is correctable by syndrome decoding using table
coincides with one of
T 's
T
if and only
coset leaders.
Sage-notebook: Syndrome decoding
Slepian array (or standard array) Not a real algorithmic tool but rather an interpretation of what happens
3-16
in syndrome decoding: The Slepian array to syndrome table arrangement of all words
Example:
∈ Qr
in an
T with coset q r × q k table:
leaders
f1 , ..., fqr
is an
A Slepian array for the Hill-code.
00000
11100
00111
11011
00001 00010 00100 01000 10000
11101
00110
11010
11110
00101
11001
11000
00011
11111
10100
01111
10011
01100
10111
01011
01010 01001
10110
01101
10001
10101
01110
10010
columns are numbered by codewords, rows by coset leaders (listed by increasing weights)
entry in row
f
c
and column
consequently row
f
is
f +c
consists of the coset
Observation: up to weight
∗
t
f +C
coset leaders are unique (incomplete syn-
drome decoding relies only on these unique coset leaders)
Error rates for syndrome decoding
Theorem 3.10. Let
C ⊆ Bn
3-17
be a linear code such that for
the number of coset leaders of weight
i
is
αi .
i = 0, 1, ..., n
Then, the probability of
correct decoding a received word by means of syndrome decoding is
Pcorr (C) =
n X
αi pi (1 − p)n−i ,
i=0
under conditions of the binary symmetric channel with bit error rate
48
p.
Proof. the only errors that can be corrected are the coset leaders
WER(C) = 1 − Pcorr
Syndrome decoding of binary Hamming codes
recall: order
r
Hamming matrix
vectors of length
H
3-18
consists of all non-zero binary
r
w.l.o.g. we order them in a way such that ith column = representation of
i
(notation:
r
bit binary
binr (i)T )
Sage-command(s):
on the other hand, the correctable errors are exactly the weight 1 vectors of length
let
ei
r
n=2 −1
denote the
ith
unity vector
(0, ..., 0, 1, 0, ..., 0), | {z } | {z } i−1
drome of
ei
is
codes.HammingCode(r,GF(2))
is based on matrices of exactly this shape
then the syn-
n−i
binr (i)T
this gives the following correction of
r
from single errors
* yT := H · rT * i := bin−1 r (y)
ˆ := r − ei * c the underlying syndrome table maps columns to corresponding unity vectors
3.5 Application to certain Reed-Solomon codes
Syndrome decoding of the "de Luxe ISBN-code"
3-19
The following example is taken from Hill's book.
consider the
[10, 8]11 -code controlled by 1 1 1 1 1 1 1 H= 1 2 3 4 5 6 7
1 8
1 9
1 10
the de Luxe ISBN code is a subcode thereof obtained by expurga11 tion : we omit all words containing digit '10' consider the transmission
c = (c1 , ..., c10 ) ; r = (r1 , ..., r10 ) = c + e
and compute the corresponding syndrome
P10 r A = H · rT = P10i=1 i B i=1 i · ri 11 removing
some codewords
49
(mod 11)
in case a single error occured, we have
e = (0, ..., 0, m, 0, ..., 0) | {z } | {z } `−1
for some location
`
10−`
m
and error magnitude
the corresponding syndrome parts are A
= m (mod 11) and B = `·m
(mod 11) thus we conclude: magnitude =
A
and location =
B/A := B · A−1
BMD decoder
3-20
given r, compute the syndrome parts
(A, B) (sums of unweighted and
of weighted symbols)
if
(A, B) = (0, 0)
assume no errors
if
(A 6= 0, B 6= 0) assume single error of magnitude A ` = B/A (to correct: replace r` by r` − A)
if
A 6= 0
and
B 6= 0
but not both:
in location
at least two errors occured
(Example: swapped symbols), do not try to correct
Remark
*
this code is an example of a decimal code: symbols are ordinary decimal digits, well-suited for check-digit systems
*
it is not linear any more, nevertheless syndrome decoding works well: this is true for any subcode of a linear code
Going further: A double-error-correcting code
consider the
[10, 6]11 code controlled by 1 1 1 ... 1 1 2 3 . . . 10 H= 1 22 32 . . . 102 1 23 33 . . . 103
3-21
(mod 11)
which is double-error-correcting (by the main lemma on linear codes: any four columns form a Vandermonde matrix, so min dist is 5)
syndrome of
r
is given by
y T = H · rT ,
so
10 10 10 10 X X X X y = (S1 , S2 , S3 , S4 ) = ( ri , iri , i2 ri , i3 ri ) i=1
Rearranging syndrome equations
50
i=1
i=1
i=1 3-22
assume two errors of magnitudes
a, b
in positions
i, j
respectively,
then
a + b = S1
(3.1)
ai + bj = S2
(3.2)
2
2
(3.3)
3
3
(3.4)
b(i − j) = iS1 − S2
(3.5)
bj(i − j) = iS2 − S3
(3.6)
ai + bj = S3 ai + bj = S4 eliminate
a, b, j
as follows:
i × (1) − (2) i × (2) − (3) i × (3) − (4)
gives
gives
gives
2
bj (i − j) = iS3 − S4
(3.7)
Localization equation
comparing
2
(6)
with
3-23
(5) × (7)
gives
(iS2 − S3 )2 = (iS1 − S2 )(iS3 − S4 ) implying
(S22 − S1 S3 )i2 + (S1 S4 − S2 S3 )i + S32 − S2 S4 = 0 if instead
a, b, i
where eleminated in similar fashion we would get
same equation with
thus:
i, j
once
i, j S1 , S2
(3.8)
j
replaced by
i
are the roots of this quadratic equation are known,
a, b
can be easily obtained from syndrom parts
Deriving a BMD decoder for this code
consider the coe cients
S32 − S2 S4
3-24
P = S22 − S1 S2 , Q = S1 S4 − S2 S3 , R ==
of the quadratic equation
observe that in case of a single error in position
a,
i
and of magnitude
we had
S1 = a, S2 = ai, S3 = ai2 , S4 = ai3 implying
P =Q=R=0
BMD-decoder Given
r
compute
*
if
*
else compute
S(r) = 0
magnitude
S(r) = (S1 , S2 , S3 , S4 )
then assume no errors
P, Q, R, if P = Q = R = 0 S1 in position S2 /S1 51
assume a single error of
*
P 6= 0, R 6= 0
else if
and if
Q2 − 4P R = T 2
T ∈ F11 ,
for some
assume two errors
*
·
in positions
·
with magnitudes
i, j =
−Q±T and 2P
b = (iS1 − S2 )/(i − j), b = S1 − b
else conclude that at least three errors occured
Example
3-25
Generalization to arbitrary finite fields
3-26
consider an arbitrary nite eld
F = Fq
q -nary
and the
code con-
trolled by a matrix
1 h0
1 h1
... ...
1 hn−1
,
where the second row consists of pairwise dierent values
∈ F∗ =
F \ {0} (this condition implies
n < q)
again, the syndrome parts
A, B
of received vectors
r
are sum of un-
weighted, weighted symbols
hence, in case of single errors the error magnitude is the error location is the position
`
with
h` = B · A
A=
P
ri
−1
A class of Reed-Solomon-codes Let
n = q − 1 and ω
3-27
be a primitive root, i.e., a eld element that generates
the unit subgroup in the following sense:
F∗ = {1, ω, ω 2 , ..., ω n−1 }. Then the code
RS2 (ω)
controlled by
H2 (ω) = is equivalent to
Polyb n,n−2
1 1
with
1 ω
1 ω2
... ...
1
ω n−1
b = (1, ω, ω 2 , . . . , ω n−1 ).
(we will inspect this relation soon a little deeper - for now please keep only in mind, that codes de ned like this are something we've already met)
BMD-decoder similar to the above
*
error magnitude =
*
error location = where we
A
logω (a), where a = B · A−1 number locations 0, 1, ..., n − 1
52
(discrete logarithm),
Sage-command(s): discrete_log
Example: Discrete log mod 11
2i (mod 11) i
1
2
4
8
5
10
9
7
3
6
0
1
2
3
4
5
6
7
8
9
2nd row = logs of numbers in top row
More Reed-Solomon-Codes
3-28
decoding idea of two-error-correcting decimal code can analogously extended to
q -nary
codes controlled by matrices like
1 1 1 1 where
ω
1 ω ω2 ω3
1 ω2 ω4 ω6
... 1 . . . ω n−1 , . . . ω 2n−2 . . . ω 3n−3
is an arbitrary primitive root in
Fq
and
n 0. A biparte graph (V ∪ R, E) is called an (α, δ)-expander S ⊆ V, |S| ≤ α|V | holds: |N (S)| > δ · |S|. ·
if for all
S, ∅ 6=
(small variable sets expand to many checks)
Expansion properties and code parameters
Theorem 3.24. If and
C = C(T )
3-52
T is a (dL , dR )-regular (α, δ)-expander with the corresponding code, then
δ ≥ dL /2
d∗ (C) > αn. In particular, Proof.
let
C 's
b αn 2 c.
error tolerance ist at least
assume some
c ∈ C(T )
of weight
|c| ≤ αn
S = {i ∈ V : ci = 1}
each check condition otherwise
h ∈ N (S)
has at least two neighbours in
S,
hcT 6= 0
hence, between
S
and
N (S)
there are
2 · |N (S)| > 2δ · |S| ≥ dL |S|
edges
but by regularity there are only
Example
dL |S|
edges connected to
S
3-53
62
Example 3.25. Following graph de nes a
(α, δ)-expander
In fact,
for
α = 2/9
and arbitrary
[9, ≥ 3, ≥ 3] δ < 3/2 (see
code, since it is a worksheet).
dim = 4, d∗ = 4.
Spielmans Bit-flip decoder
consider a given
3-54
(α, δ)-expanding Tanner graph on variables x1 , ..., xn h1 , ..., hr for the corresponding code C = C(T )
and check conditions
given
i ∈ {1, ..., n} let N (i) denote the HN (i) the corresponding N (i))
conditions and in
Iterative majority decoding of initialize current vector :=
i
r
r
iterate while (there is i, such that ip bit
set of neighboured check matrix (rows = conditions
maj(HN (i) )rT = 1)12 :
in current vector
nalize return
ˆ := c
current vector
Observations This algorithm terminates with a codeword, since every iteration reduces the number of unsatis ed check conditions. But bit ipping may introduce new errors, therefore we need stronger assumptions to guarrantee
d(ˆ c, r) ≤ t∗ .
Correctness of the algorithm
Theorem 3.26. If
T (C)
is an
(α, δ)-expander
3-55
with
δ ≥ 43 dL 13 ,
then the
αn bit ipping decoder corrects all errors up to weight b 2 c.
Proof given
r ↔ e,
let
u :=
number of unsatis ed equations and
as the algorithm proceeds, the aim is to reach
t = |e|
e is updated as well as current state (u, t); (u, t) = (0, 0)
12 x occurs in more unsatis ed than satis ed equations i 13 recall: weaker condition δ ≥ d /2 implies d∗ C(T ) > αn L
63
by regularity the
t
dL · t connections to check u are unsatis ed and let s be dL t ≥ u + s
corrupt variables have
conditions, of which (by assumption) the number of satis ed ones, hence
in case
t < b αn 2 c,
we have in particular
u + s > δt ≥
t < αn,
so by expansion
3 dL t 4
Proof (continued)
3-56
any satis ed check equation connected to a corrupt variable must be connected to a 2nd corrupt variable (else would be unsatis ed)
further, any unsatis ed check condition contains at least one corrupt variable
dL t ≥ u + 2s = (u + s) + s >
3 dL t + s 4
dL t 4 and since more than half of the connected checks are d t unsatis ed, we have u > L 2
so
s
dL t dL αn ≥ 2 2
but this contradicts the fact that initially we have
u < dL t < dL and
u
αn 2
is decreased in every iteration
Remarks on existence of good expanders
for small
n
it is not too di cult to nd regular expanders with not
too bad expansion (see exercises)
64
3-58
for large (or increasing)
n
only few results are known, e.g., the con-
struction of Ramanujan-graphs = bipartite graphs, whose adjacency matrices have certain eigenvalue properties a very complex construction found in the 1980ies
some hope is based on Monte-Carlo methods: Theorem 3.27 (Pinsker 1973). A random regular bipartite graph is a good expander with high probability. Example: The DVBT2-standard makes use of experimentally found LDPC codes.
drawback of randomly chosen graphs:
computationally intensive
proof of expansion properties
famous result of Sipser and Spielman 1996: linear time construction of linear time encodable and decodable LDPC codes!!
however this is still impractical, since the construction leads to codelength of order Lecture 10)
14
≥ 1064
(Madhu Sudan, Essential Coding Theory,
3.9 Complexity of decoding
Setup 1: Fixed code
3-59
C = ker H, H ∈ Qr×n , we want to express n eort to decode r ∈ Q by syndrome decoding in
consider a xed code the computational terms of
n
optimistic assumption: syndrome table
*
recall:
f, *
T
maps
y
T : Qr → Qn
to a coset leader = minimum weight vector
that has syndrom
Hf T = yT
without further knowledge about the structure of of
T
as a size
qr
is known
C
we can think
table with syndromes on the left hand side and
coset leaders on the right
*
let the syndromes be lexicographically sorted
decoding
*
r
using
T
amounts to compute and return
arithmetic operations in
14 Take
y T = H · rT the eld Q
computing the syndrome
takes
r − T (H · rT ):
O(n · r) = O(n2 )
this gure with caution, currently I cannot nd the citation.
What I remember
is this: Sipser/Spielman'96 suggested a linear-time randomized algorithm that nds w.h.p. an LDPC code with linear-time codec and reasonable length (however, the cost for checking the expansion property is prohibitively large). Derandomizing this construction leads to the tremendous increase in length and probably also worse code parameters
65
*
looking up
y in T
*
correction
O(n) eld operations using binary search O(q r ) = O(r) = O(n) ops)
takes
(binary search takes
r 7→ r − T (y)
takes another
altogether, in setup 1 decoding takes
O(n)
2
O(n )
ops
time
(syndrome computing being the most time consuming step)
Setup 2: Code as part of the input
3-60
De nition 3.28. The decoding problem consists in the following Input: generator word
n
r∈Q
G
matrix of a linear code
[The
list decoding problem
c ∈ C,
received
t≥0
, closeness parameter
Output: some codeword
C ⊆ Qn ,
such that
requires to output a list of
d(c, r) ≤ t
all c ∈ C ,
such that
d(c, r) ≤ t]
Decoding is a search problem, the corresponding decision problem is:
De nition 3.29. The nearest codeword problem (NCP): Input: matrix
G,
closeness parameter
Output: TRUE, i there is some
t≥0
c ∈ spanG,
such that
d(c, r) ≤ t Theorem 3.30. NCP is NP-complete.
A well known NP-complete problem
3-61
obviously NCP is in NP (guess&check) for the completeness proof we use reduction from the following NPcomplete problem
De nition 3.31. The MAX-CUT problem: Input: integer parameter rected graph (rows
m and incidence matrix I
of an undi-
(V, E)
nodes, columns
edges,
Iv,e = 1
i
v
is incident to
e)
Output: TRUE, i (V, E) contains a cut set S ⊆ V of size ≥ m (cut set size of S := |{e ∈ E : e connects a node in S to a node outside S}|)
Proof
given
3-62
m
and
I
we de ne generator matrix
closeness parameter
* G := I ∈ Fk×n 2 * r := 1n
and
G,
received word
t:
(code dimension
t := n − m
66
k = |V |,
length
n = |E|)
r
and
Claim:
(m, I) ∈ MAX-CUT
if and only if
(G, r, t) ∈ NCP
Proof:
*
to
*
let
S ⊆ V consider its characteristic by si = 1 if and only if i ∈ S e = {v1 , v2 } ∈ E
vector
s ∈ {0, 1}k ,
de ned
be some edge, then
X
T (sG)e = s · I∗e =
sv · Iv,e
(mod 2)
v is incident to e
*
because there are only two nodes incident to each
|e ∩ S| (mod 2), which S ⇔ e is a cutting edge
sum equals belongs to
*
⇔
e,
the latter
exactly one of
v1 , v2
c ∈ spanG in distance ≤ n − m of 1n ≥m
consequently, there is some
⇔
is 1
there is a cut of size
Complexity of syndrome decoding
3-63
decoding problem remains di cult, if code is given by check matrix (while in setup 1 the syndrome table is delivered by R& D division, we now have to care about it ourselves!)
De nition 3.32. The maximum likelihood decoding problem (MLD): Input:
H ∈ Qr×n , y ∈ Qr , w > 0
Output: TRUE, i ∃e ∈ Q with |e| Theorem 3.33. MLD is NP-complete.
≤ w, H · eT = yT
Proof idea reduction from the NP-complete 3-dimensional matching problem: Given
t>0
and
T ⊆ X × Y × Z , decide whether ∃M ⊆ T, |M | ≥ t, M coincide in one component.
such that
no two elements of
Upper bounding the minimum distance
3-64
De nition 3.34. The minimum distance problem MD: Input:
H ∈ Qr×n , w > 0
Output: TRUE, if ∃c ∈ ker H ∧ |c| Theorem 3.35. MD is NP-complete.
≤w
Remark MD is MLD with xed syndrome MLD)
67
y = 0r
(looks easier than general
completeness proof only in 1997
Approximating the minimum distance
3-65
Theorem 3.36. There is no polynomial time algorithm, that given
1, H ∈ Qr×n
computes an approximation
d
α>
such that
d ≤ d∗ (ker H) ≤ αd, unless
P = NP.
Structured codes
3-66
NP-completeness results on NCP are worst case statements on all linear codes
perhaps things become easier, if we restrict attention to special structured codes?
however, e.g., the family of polynomial codes is rich enough to feature hard instances
Theorem 3.37 (Guruswami, Vardy 2004). The following problem is NPcomplete:
Input: degree bound
t > 0,
k > 0, sample r ∈ Qn
vector
b ∈ Qn ,
tolerance
target vector
Output: TRUE, if
∃c ∈ Polyb n,k
such that
d(r, c) ≤ t.
3.10 Exercises
1. The bar-product (or Plotkin-sum, or . . . - several names for the same
(n, M1 , d1 )-code C1 C := C1 |C2 de ned by
thing) of binary is the code
(n, M2 , d2 )-code C2
and binary
C = {(u|u + v)|u ∈ C1 , v ∈ C2 } prove that
C
is a
prove that
C
is linear, if
(2n, M1 M2 , d)-code, C1 , C2
where
d = min{2d1 , d2 }
are.
2. Reed-Muller-codes can be recursively de ned as follows:
RM(0, m) = {02m , 12m }
RM(m, m) = {0, 1}2 2m )
RM(r + 1, m + 1) = RM(r + 1, m)|RM(r, m)
m
(binary repetition code of length
2m )
(the full binary ambient space of length
68
for
m≥r≥0
By previous exercise each rameters of
RM(r, m):
RM(, )-code
3. The direct product of binary codes whose rows are
C1
is linear. Determine the pa-
length, dimension, and minimum distance.
C1 , C2
consists of all matrices
codewords, and whose columns are
C2
codewords.
prove that the minimum distance of the direct product code is the product of the minimum distances of both codes
consider the cross parity check code = direct product of two simple parity check codes. What is its minimum distance? Design a corresponding decoder.
4
Existence and construction of good codes
4.1 Combinatorial upper bounds
Largest codesize
4-1
a good code of given length has large size and large minimum distance
these are con icting aims: the ambient space
Qn
is too small to
contain very many very distant codewords
which code sizes are achievable?
Aq (n, d) denotes the largest code size of a q -nary code of length n and d
minimum distance
Trivial observations
1.
Aq (n, 1) = q n , Aq (n, n) = q
2.
Aq (n, d1 ) ≤ Aq (n, d2 )
if
d1 ≥ d2
3.
Aq1 (n, d) ≤ Aq2 (n, d)
if
q1 ≤ q2
Some values for the binary case
69
4-2
n
d=3
d=5
d=7
5
4
2
6
8
2
7
16
2
2
8
20
4
2
9
40
6
2
10
72
12
2
11
144
24
4
12
256
32
4
13
512
64
8
14
1024
128
16
15
2048
256
32
Sage-notebook: Largest binary codes
Odd and even minimum distances
the table lists only odd values of
4-3
d∗
this is su cient in the binary case by the following lemma Lemma 4.1. If d > 1 is odd, than A2 (n + 1, d + 1) = A2 (n, d). Proof. Exercise.
Hint: Use code extension and puncturing, respectively.
The ball packing bound (Hamming bound)
Theorem 4.2. For
d = 2t + 1
4-4
holds
Aq (n, d) ≤ q n /Volq (n, t) , Pt n i where Volq (n, t) := i=0 i (q − 1) is the volume n Hamming space Q .
of a radius
t
ball in
Sage-command(s): volume_hamming(n,q,r)
Proof.
let
2t + 1
M
be the maximum size of a length
n,
minimum distance
hamming_upper_bound(n,q,d)
code
if codewords are spaced away by distance
2t + 1, the balls of radius t ≤ t) are disjoint
around codewords (sets of words in distance
M
such balls t disjointly into
Qn ,
hence the bound
Example: Binary Hamming codes meet the upper bound the bound cannot be improved.
Singleton's bound
4-5
Theorem 4.3.
Aq (n, d) ≤ q n−d+1 . 70
Sage-command(s):
take the
Proof.
singleton_upper_bound(n,q,d)
M
minimum distance
codewords of an optimum (largest size) code of
d
and cancel the rst
resulting words will still have distance so we still have hence
M
(d − 1)
components
≥1
words and these live inside
Qn−(d−1)
M ≤ q n−d+1
Remark the cancelling operation is called puncturing
Sage-command(s):
it is the same as projecting codewords onto the remaining positions Example:
Polyn,k
punctured(L)
meets the Singleton bound the bound cannot be
improved.
Plotkin's bound
4-6
notation:
θ := 1 −
1 q−1 = q q
q = 2 we have θ = 12 Theorem 4.4. 1. For d > θn holds Aq (n, d) ≤ and in particular for q = 2: in particular, for
A2 (n, d) ≤
Sage-command(s):
d d−θn
plotkin_upper_bound(n,q,d)
2d . 2d − n
d d > θ(n − `), ` ∈ N holds Aq (n, d) ≤ q ` d−θ(n−`) and in particular for q = 2:
2. For
A2 (n, d) ≤
2`+1 d . 2d − n + `
Proof of first part (only binary case)
let
M
C of length n d(c 1 , c2 ) c1 ,c2 ∈C
be the maximum size of code
be its total distance
D=
P
71
4-7
and
d∗ = d
and
D
(1) there are
M (M − 1)
distances in the sum, hence
d≤
D M (M − 1)
(the RHS is the average distance between codewords)
to estimate
D
count separately numbers
fering in position
let
B(i)
of codeword pairs dif-
i
M0 = M0 (i), M1 = M1 (i) i
= numbers of codewords with 0, 1 in
bit position
B(i) = M0 (M − M0 ) + M1 (M − M1 ) = M 2 − (M02 + M12 ) 1 1 ≤ M 2 − (M0 + M1 )2 = M 2 2 2 (equality holds i code is balanced in position (2) consequently
D=
Pn
i=1
B(i) ≤
i,
i.e.
M0 = M1 )
n 2M
combining with inequality (1) this gives
d ≤
nM 2 , which by some
rearrangements implies the statement
Proof of second part follows from first ...
4-8
` times: De nition 4.5. Let C be an (n, M, d)-code. For a xed position i ∈ {1, 2, ..., n} consider the subcodes Ca consisting of codewords c ∈ C with ci = a. By cancelling the ith position in all codewords of Ca , we obtain a ... by applying the following operation
Sage-command(s):
shortened code. The largest shortened code has size
≥ M/q ,
which by construction is a
shortened(L)
(n − 1, ≥ M/q, d)-code. Corollary 4.6.
Aq (n, d) ≤ q ` Aq (n − `, d). Remark Plotkin's bound achieves equality only for (1) equidistant codes, that (2) are additionally balanced in each position, i.e.,
#C0 = #C1 = ....
(see
proof of rst part) Example: Simplex codes Plotkin's bound cannot be improved.
Remark on applicability
4-9
Hamming's bound works best for small
∗
d
reason:
*
Hamming's bound is sharp only for perfect code
72
*
it is known since the 1970ies, that any perfect code has minimum distance 3, 5 or 7 others don't exist
Plotkin's bound on the other hand works best for Simplex codes, i.e., large
d∗
4.2 Combinatorial lower bounds
Upper and lower bounds
4-10
Hamming, Singleton, Plotkin are upper bounds: no larger codes exist for certain parameter settings
(n, d)
we are interested in the best =
lowest upper bound
next, we want to see lower bounds on
Aq (n, d),
i.e., constructions of
as large as possible codes with certain parameter settings
(n, d)
Gilbert's construction
4-11
simple greedy idea: add codewords from a list, as long as distance condition is satis ed
Gilbert's algorithm Input list L = {x1 , x2 , ...} ⊆ Qn , distance bound Output code C ⊆ Qn such that d∗ ≥ d Initialize
C=∅
Iterate while
L 6= ∅
c :=
C := C ∪ {c}
L := L \ Bx (d − 1)
rst element of
Finalize return
L
(apple biting)
C
this algorithm obviously returns an the number at least
d
M of codewords #L/Volq (n, d − 1)
(n, M, ≥ d)-code
equals the number of iterations and is
Gilbert's bound
4-12
Theorem 4.7.
Aq (n, d) ≥ q n /Volq (n, d − 1) . Interesting facts
73
If
L is lexicographically ordered the algorithm produces a linear code
If
n = 2r − 1
and
d=3
the corresponding Code is a Hamming code!
Sage-notebook: Gilbert's algorithm Exercise
4-13
Surf the internet for the term Lexicode and explain this notion. Try to prove (or at least to convince), that lexicodes are linear codes. modify the Sage-notebook Gilbert's algorithm to yield lexicodes.
Varshamov's construction
Recall:
d∗ =
4-14
minimum size of a linear dependent set of columns
Idea: construct check matrix by selecting column vectors
Qr ,
H∗j
H ∈ Qr×n
of check matrix
such that any
d−1
h1 , h2 , ... ∈
of them are linear independent
this means: if
S = {h1 , ..., hj−1 } is the set of already chosen columns, hj we have to avoid all linear combinations of at most vectors in S
then to choose
d−2 let
of the
Ld−2 (S)
denote the set of all such linear combinations
the size of
Ld−2 (S) is at most j−1 j−1 j−1 2 N (j) = 1 + (q − 1) + (q − 1) + ... + (q − 1)d−2 1 2 d−2 = Volq (j − 1, d − 2)
consequently: a next column can be chosen, as long as
N (n − 1) < q r we can choose n columns matrix H controls a [n, n − r, d]q -code
in particular, if resulting check
N (j) < q r and the
The algorithm
4-15
Construct check matrix
Sage-notebook: Varshamov's construction
Initialize
S=∅
Iterate
E := Qr \ Ld−2 (S)
choose
∈ Qr×n for code of minimum distance d
h∈E
S := S ∪ {h}
until
E=∅ 74
Finalize return
H := (hT |h ∈ S)
Exercise Implement Varshamov's algorithm in Sage.
Varshamov bound and comparison to Gilbert bound
4-16
Theorem 4.8.
Aq (n, d) ≥ max{q k |q k < Proof.
qn }. Volq (n − 1, d − 2)
N (n − 1) < q r ⇔ q k < q n /Volq (n − 1, d − 2)
for
r = n − k.
Comparison Varshamov's algorithm is guaranteed to give a linear code (this is not the case for Gilbert's algorithm)
the order
Ω(q n /nd−2 )
of Varshamov's bound is slightly better then
that of Gilbert's bound:
Ω(q n /nd−1 )
4.3 Asymptotic bounds
Limit rate
let
p < 1/2
4-17
be the symbol error rate in a channel
the expected number
t = np
of symbol errors in a transmission of
symbols is acceptable for codes with
n
d∗ ≥ np + 1
the maximum information rate for such codes is
1 logq Aq (n, d) n
aq (n, d) :=
we are interested in the asymptotic behaviour of this quantity in case of xed normalized distance
δ=
1 As it is unknown if the sequence n it's upper limit:
d n (same as distance rate
logq Aq (n, δn)
D)
converges, we consider
De nition 4.9. The limit rate of q -nary codes with normalized distance
δ
is de ned as
αq (δ) := lim sup n→∞
75
1 logq Aq (n, δn). n
αq (δ)
is the limit information rate achievable by codes whose codewords
dier in at least a fraction
δ
of all their positions.
Asymptotic upper bounds
4-18
by taking limits in the combinatorial upper bounds, we obtain corresponding asymptotic bounds:
Singleton
αq (δ) ≤ 1 − δ ( 0 Plotkin αq (δ) ≤ 1 − δ/θ,
, if
δ>θ
else.
[Proofs left as exercise.]
The entropy function
4-19
for stating the asymptotics of Hamming's and Gilbert's/Varshamov's bound (both have the same asymptotics) we need the following function de nition
De nition 4.10. The binary entropy function is the function
h(p) = −p log2 p − (1 − p) log2 (1 − p). The
q -nary
entropy function is
hq (p) = −p logq p − (1 − p) logq (1 − p) + p logq (q − 1). convention:
0 log 0 = 0
maximum is taken at
p=θ
(recall:
θ =1−
1 q)
Sage-notebook: Entropy function
Entropy and volume of balls in Hamming space
Lemma 4.11. For xed
δ, 0 ≤ δ < θ
4-20
holds
lim logq Volq (n, bδnc) = lim logq Volq (n, dδne) = hq (δ).
n→∞
n→∞
analytically: make use of Tschebysche 's inequality or so (bor-
idea.
ing)
information theoretically (binary case only, can be extended):
*
conveyed by the fact that centered at
*
r ∈ Bn is radius ≈ δn
Q: How many bits of information about a random
r
falls inside a ball of
0n ?
A: The same amount that is conveyed by the message: at most a fraction
≈δ
of
n
ipped coins showed up heads, that is
76
h(δ).
For later use: An interesting inequality
4-21
A related statement for the binary case (which will be used later) is:
Sage-notebook: Volume of balls
Lemma 4.12. If
0 ≤ δ ≤ 1/2,
then
Vol2 (n, bδnc) ≤ 2n·h(δ) . There is a (non-boring) elementary short proof of this fact:
Proof. For sake of simplicity let
δn
be integer. Then
1 = (δ + (1 − δ))n ≥
δn X n i=0
i
δ i (1 − δ)n−i
δn δn X n δ 1−δ i i=0 δn X n δn (1−δ)n . = δ (1 − δ) i i=0 δn X n ≥ . i i=0 ≥ (1 − δ)n
Hence
δ −δn (1 − δ)−(1−δ)n
Taking logarithms now gives the inequality.
Further asymptotic bounds
4-22
Applying the relation between ball volume and entropy we obtain
Hamming
αq (δ) ≤ 1 − hq (δ/2)
Gilbert-Varshamov
(upper bound)
αq (δ) ≥ 1 − hq (δ)
(lower bound)
Another asymptotic upper bound is the so-called linear programming bound (reason for this naming: proof relies on MacWilliams identity and duality concepts from linear programming), here stated only for the binary case:
LP-bound
αq (δ) ≤ h( 21 −
p δ(1 − δ))
Comparison of the bounds
4-23
Sage-notebook: Asymptotic code bounds
77
all binary code families
Cn (n, Mn , dn ) known today converge in the all limit points of (R(Cn ) , D(Cn )) are
grey area (more precisely: inside this area)
parameters
(R, D)
of the code families we studied so far (Hamming,
Hadamard, simplex, ...) converge to some point on one axis of the diagramm (see Sage notebook)
no code familiy can exist, that converges to some point beyond any of the coloured lines
4.4 Good codes
Motivation
4-24
De nition 4.13. Let
δ > 0, ρ > 0
be xed values.
Cn (n, Mn , dn ) such that n is called (δ, ρ)-good.
A code family nitely many
A family of codes that is
dn n
≥ δ
and
logq Mn n
≥ ρ
for in-
(δ, ρ)-good for some δ, ρ is called an asymptotically
good code (note the singular!). asymptotically good codes allow reliable communication:
at a guaranteed speed (fraction
≥ ρ
of each codeword usable for
message encoding) and
up to a certain noise level (fraction
≤ δ/2
of wrong symbols is cor-
rectable)
Good codes exist
4-25
78
Theorem 4.14. Let there exists a
0 < δ < θ and ρ = 1 − hq (δ). Then for every ε > 0 (δ 0 , ρ0 )-good code family, where (δ, ρ) = (δ − ε, ρ − ε).
let
Proof.
dn = bδnc,
hence
δ−ε
(1 + ε)np ≤ eε 3 .
If
i=1
let
Cn
be an
WER(Cn , p)
(n, ≥ 2ρn , ≥ δm)-code is upper bounded by the WER of the bounded distance
decoder (BDD) that decodes
b δn−1 2 c
r
to
c ∈ Cn
only if
d(r, c) ≤ dn :=
and fails otherwise
Error exponent
(Proof continued). if
let
Xi = 1
4-38
p
0 there is some n such that there n and information rate Rn > 1−h(p)−ε (E, D) such that WER(E, D) ≤ ε.
and any
exists a binary code of length and corresponding codec
Some ideas of the proof
*
for
*
codebook
(n, k) consider a random encoding scheme E : {0, 1}k 3 s 7→ c ∈ {0, 1}n (c is chosen randomly - strange, isn't it?) E({0, 1}k ) is the promised code, decoder D is described
below
* D
is the following incomplete decoder:
D(r) =
( c=
the unique close codeword
else,
FAIL
where c is close codeword means
*
d(r, c) ≤ n(p + ε)
15 that appropriate then it can be shown ensure that
if it exists
k/n > 1 − h(p) − ε
and also
(n, k) can be found WER(E, D) < ε
to
Remarks on the channel coding theorem
it turns out, that the error bound is exponentially small: there are constants
A, B > 0
such that
WER(E, D) ≤ Ae−Bn we obtained a similar thing already for asymptotically good codes, but this relied on bounded minimum distance decoding and works only for
15 using
p < δ/2
tools like Chebyshev's inequality, union bound and also the earlier considered bound
on volume of balls in Hamming space
91
5-7
6
Covering codes
6.1
q -nary
Hamming codes
Motivation
6-1
De nition 6.1. A
if the cor-
rection balls, i.e., the sets
completely
ll-up the ambient
q -nary (n, M, d) code C is called perfect, Bt (c) with c ∈ C and t = d d−1 2 e n space Q .
Equivalent condition is the ball packing equation:
t X n q =M· (q − 1)i . i i=0 n
Qn itself is perfect (with t = 0) and binary repetition codes of length n = 2t + 1 are perfect. Also singular codes like {0} can be considered perfect (with t = n). These examples are
Example 6.2. The ambient space
the trivial perfect codes.
binary Hamming codes are non-trivial perfect codes with
t=1
the ball packing equation requires odd d, so non-trivial perfect codes have
d∗ = 3
Question Perfect codes are interesting combinatorial objects. Are there further non-trivial perfect codes?
1st try
6-2
Aim: want to de ne q -nary Hamming codes for Idea: construct check matrix non-zero columns Example: Let
H
q>2
by including one-after-another all
∈ Qr
Q = F3 . Consider code C controled by 0 0 1 1 1 2 2 2 H= ∈ F2×8 3 1 2 0 1 2 0 1 2
By main lemma on minimum distance:
d∗ (C) = 2
(e.g., rst two columns
are linear dependent), which is too small for being perfect.
1st try didn't work because we didn't rule out multiples of a non-zero column
this eect cannot be observed for binary codes
92
q -nary
Hamming codes
Compute check matrix
6-3
Hr (Q) of q -nary Hamming code of order
r Initialize
Iterate while
V = Qr \ {0}, H =
h :=
(empty
r×0
matrix)
V 6= ∅: rst vector in
V T
update
H := (H|h )
Sage-command(s):
update
V := V \ {λh|λ ∈ Q∗ }
H.augment(h.column()))
Finalize return
Hr (Q) := H
Observation each iteration removes since initially
n= q -nary
q r −1 q−1
#V = q r − 1,
vectors from
V
the algorithm returns
Hr ∈ Qr×n ,
where
Hamming codes are perfect
Theorem 6.3. tance
q−1
HAMr (q) := ker Hr (Q)
6-4
is a perfect code of minimum dis-
Sage-command(s):
3.
HammingCode(r,GF(q))
Proof.
by construction no two columns of
dependent, so
H = Hr (Q)
d∗ ≥ 3
also by construction: for any distinct columns
H∗i + H∗j
are linear
is a column of
H,
so
H∗i , H∗j
a multiple of
d∗ ≤ 3
the ball packing equation can be checked by simple calculations
Examples
6-5
0 1 1 1 ∈ F2×4 3 1 0 1 2 0 1 1 1 1 1 1 H2 (F11 ) = 1 0 1 2 3 4 5
H2 (F3 ) =
1 6
1 7
1 8
1 9
1 10
(Observe the connection to the de Luxe ISBN-code).
93
∈ F2×11 11
6.2 Golay codes
The binary Golay-code
this is a binary perfect
6-6
The ternary Golay-code
this is a ternary perfect
Sage-notebook: Golay codes are perfect
[23, 12, 7]-code
[11, 6, 5]-code
6-7
More on Golay-codes
6-8
these where rst published and their properties postulated in a less than one page paper by Marcel Golay (1949)
other results in this paper:
*
construction and properties
*
some insight into which parameter combinations at all allow a
q -nary
Hamming codes
solution to the ball-packing equation
Golay codes have a very rich structure, they are of interest also to group theorists
will (hopefully) come back to these codes later when we study cyclic codes
6.3 Classi cation of perfect codes
Parameters of perfect codes
6-9
are there further perfect codes? natural approach to answering the question: start with looking for integer solutions
(n, M, d)
to the ball packing equation
except from the above code parameters Golay (1949) mentioned in his famous paper the candidate tuple
(90, 278 , 5)2
and proved, that
no code with these parameters exists
1967 van Lint did a computer search for other integer solutions to the equation and found no one in the range
n ≤ 1000, q ≤ 100
(other
extensive searches lead to the same result)
this and combinatorial investigations lead to the conjecture, that Hamming's and Golay codes are up to equivalence the only possible perfect codes (except trivial ones)
this was disproved: codes non-equivalent to but with same parameters as Hamming's or Golay's codes where constructed in the late 1960ies
94
Sage-notebook: The ball packing equation
BUT: a weakened conjecture turned out to be true: Theorem 6.4 (Tietäväinen 1972). Let q be a prime power. Then the only
(M, n, d)q of non-trivial perfect codes are the parameters of Hamming or Golay codes. possible parameters
6.4 Covering radius
A different view on codewords
6-10
consider a code of minimum distance
d∗ = 2t + 1
imagine the codewords as centers of balloons being simultaneously in ated at the same speed
t
corresponds to the time just before the baloons touch
if the baloons were in ated even further (with intersection impossible in reality), there is some moment, where the whole ambient space is contained in the union of the baloons, the corresponding radius
rcov
of the baloons is called covering radius:
rcov C = min{` : Qn ⊆
[
B` (c)}
c∈C
Properties
6-11
obviously for any length
n
code holds
d∗ − 1 ≤ rcov ≤ n 2 with equality (on the left side) in case of perfect codes
rcov
has some features reminding of properties of the correction ra-
dius:
Theorem 6.5. If trolled by
H,
H ∈ Qr×n
rcov = min{`|∀x ∈ Qn : H·xT
and
C = ker H
is the linear code con-
then is a linear combination of
`
columns of
Good Covering codes
H}.
6-12
a good covering code is one, that has small number words and small(!) covering radius
good covering codes exist:
95
M = qk
of code-
Theorem 6.6. For
0 < ρ < 1/2
r = ρn holds: if n rcov (C) ≤ r and |C| ≤
and integer
isp large enough, there is a binary code with n nρ(1 − ρ) · 2n(1−h(ρ)) .
sketch of proof
*
n · S = n · 2n /Vol codewords (Vol denotes radius r ball in n dimensional Hamming space)
randomly choose volume of a
*
the probability that some xed
*
the probability that
*
x
x
is not covered is
the
1 − 1/S
is not contained in any of the balls is
(1 − 1/S)nS ≤ e−n the probability that some
n→∞
x is not covered by the balls is therefore
≤ 2n /en → 0 *
Vol can also be lower estimated in terms of the entropy fct, hence the statement
Application 1: Lossy data compression schemes
lossy DCS
(E, D):
require only
6-13
D(E(s)) ≈ s
idea:
*
c
*
∈ Qn x some near that, d(c, x) ≤ rcov (C)
for any vector
=x
such
x
instead of transmitting
codeword, i.e. a codeword
x (n symbols), Alice only transmits the
corresponding near codeword, which can be speci ed by only
k
symbols (she determines the close codeword and decodes (!) it to the corresponding
*
k
message bits, which she sends)
assuming noiseless transmission Bob expands the received message to the
n
symbols of the codeword (he encodes (!) the mes-
sage), which is perhaps not identical to the original
x but at least
close to it
Alices job in this game is called quantization, Bob's is called reconstruction
works especialy well for perfect codes, since quantization points can be uniquely assigned
Example
consider
6-14
C = {000, 111}
96
*
s ∈ {0, 1}3 , decodes the quantized version bit of s = (s1 , s2 , s3 )) to b, which she sends
Alice chooses some
sss (s
= majority
(3 bits reduced to 1 bit data compression rate = 3)
*
Bob reconstructs
s
into
sss
and at most 1 bit will be wrong!
In 4 equally-probable cases a total of 3 wrong bits amounts to an average error of 3/12 = 1/4 per bit
if
C
is the (7,4) Hamming code, the compression rate is 7/4 and the
average error per bit is 1/8
if C is the binary Golay code, the compression rate is
23/12 ≈ 2
and
in each transmission at most 3 bits (=1/8 of all bits); on average less than 12% of the bits are wrong(!)
Application idea 2: Genetic algorithms
6-15
evolutionary algorithms are a class of heuristics to solve optimization problems
starting from a set of sample solutions (initial population) the idea is to iteratively improve the solutions up to some satisfactory threshold value
to this end, in each round the current population is mixed (e.g., swapping substrings of members) and the new optimization value is determined together with adapted mixing parameters
if the initial population has bad coverage of the whole solution space, the algorithm may quickly run into a local extreme
a good idea seems to be to start with a good covering code (small set of solutions, at least one of them being close to an optimum)
Application 3: Search algorithms
6-16
similar to optimization by genetic algorithms, but here we want to nd some solution to a decision problem
similar heuristics: starting from a good covering code change the population of candidate solutions
will study this in more detail in terms of the following Example: nd satisfying assignment to boolean formula
Satisfyability
6-17
De nition 6.7. A
k -conjunctive normal form (k -cnf ) over x1 , ..., xn is a F = C1 ∧ C2 ∧ ... ∧ Ct , where every Ci is a k -clause, i.e., of shape zi1 ∨ ... ∨ zim with m ≤ k and each zi is a literal (a variable xj or a negated variable ¬xj ). formula
97
a variable assignment
j
is some
* xj
F
satis es
F , if for each C = Ci
there
such that
is contained in
* ¬xj
b ∈ {0, 1}n C
and
C
is contained in
bj = 1
and
OR
bj = 0
is called satis able if there is a satisfying assignment
the satis ability problem (given
F,
determine if its satis able) is a
well known NP-complete problem
brute force search through all possible assignments solves the problem but using runtime poly(n)
· 2n
using covering codes we can reduce the runtime to poly(n) · 2αn , for some
α0
and therefore
n − k < n.
Generator polynomial proof (1)
1. construction of Let
g(x)
7-8
g(x) and divisor property
= the monic greatest common divisor of
102
{ci (x)|...}.
ai (x)
Then there exist polynomials
g(x)
=
X
such that
ai (x)ci (x)
i
! X
=
mod (xn − 1),
ai (x)ci (x)
since
deg g(x) < n
i
=
X
mod (xn − 1)) .
(ai (x)ci (x)
i The nal expression shows that
g(x) is a linear combination of codewords.
By linearity it is a codeword, and by construction it's polynomial divides all other codeword polynomials.
2. uniqueness If there were two generator polynomials, each would be a divisor of the other, and since both are monic, the ratio is 1.
Generator polynomial proof (2)
3. degree is Let
7-9
n−k
r = deg g(x).
If
deg m(x) < n−r then deg m(x)g(x) < n, so m(x)g(x) c(x) = m(x), deg m(x) < n − r.
is a codeword. Conversely (as seen), every codeword is of shape
m(x)g(x)
for some
So there are
q n−r = q k
codewords
⇒ n − r = k ⇒ r = n − k.
4. generator polynomial divides the cyclic polynomial The right cyclic shift of
g (k) (x)
g(x)
by
k
positions is a codeword:
=
(1, 0, . . . , 0, g0 , g1 , . . . , gn−k−1 )
=
1 + g0 xk + g1 xk+1 + . . . + gn−k−1 xn−1 + xn − xn
= xk g(x) − (xn − 1) Therefore
xn − 1 = xk g(x) − g (k) (x) is a sum of multiples of
g(x)
and so itself is a multiple of
g(x).
Message polynomial
7-10
to summarize: every codeword is of shape nomial
m(x)g(x)
for some poly-
m(x) = m0 + m1 x + ...mk−1 xk−1
we can use this as an encoder:
*
for message
...mk−1 x
k−1
m = (m0 , ..., mk−1 ) ∈ Qk
let
m(x) = m0 + m1 x +
be the corresponding message polynomial
103
*
we encode
m
by the vector
c
of coe cients of the polynomial
m(x)g(x) obviously, this is a linear encoding (sums/scalar multiples of messages are encoded by sums/scalar multiples of codewords)
we now study a matrix representation of this encoding
Nonsystematic generator matrix
let
C
be an
(n, k)
7-11
cyclic code generated by
g(x)
of degree
r =n−k
xi g(x)(i = 0, 1, . . . , k −1) form a basis for C(x), since every codeword m(x)g(x) = m0 g(x) + m1 xg(x) + . . . + mk−1 xk−1 g(x) is a linear combination of
xi g(x)
consequently, the matrix with row polynomials tematic ) generator matrix for
C:
g(x) g0 xg(x) 0 .. . . G1 = =. . k−2 x g(x) 0 0 xk−1 g(x)
where
gr = 1
every row of
xi g(x) is a (nonsys-
g1 g0
... g1
gr ...
. . .
..
..
0 0
... ...
.
.
g0 0
0 gr ... g1 g0
0 0
... ...
..
..
.
... g1
.
gr ...
0 0 . . , . 0 gr
(generator poly is monic)
G1
is a right (cyclic) shift of the rst row
i.e., each message symbol produces the same encoder output, but scaled and time delayed
Will see: by
G1 leads to simple encoders using polynomial multiplication
g(x)
Examples of binary cyclic codes (1)
the singular the
7-12
(n, 0) code consisting of the zero vector: gn (x) = xn − 1.
(n, n) code of all n-tuples is another trivial cyclic code: g0 (x) = 1
(see rst part of Sage notebook)
the
(n, 0)
and
(n, n)
codes are duals, observe:
n
g0 (x)gn (x) = x − 1
simple parity-check codes are cyclic, nonsystematic generator matrix:
1 1 0 0 1 1 .. .. .. . . . 0 0 0 104
... ...
0 0
..
.
. . .
...
0
0 0 . . , . . . . 1 1
0 0
Sage-notebook: Examples of cyclic codes
so the generator polynomial is
x−1
g1 (x) = x + 1
(which is the same as
in the binary case)
repetition codes have generator matrix
G= 1 hence
the
1
1
...
gn−1 (x) = 1+x+. . .+xn−1
1
1
1 ,
(see second part of Sage notebook)
(n, n − 1) SPC code and the (n, 1) g1 (x)gn−1 (x) = xn − 1
repetition code are duals,
observe:
Systematic generator matrix (1)
7-13
recall: systematic encoder copies message digits to consecutive positions in codeword
Convention: most signi cant digits are transmitted rst, like reading the codeword right-to-left:
cn−1 , cn−2 , . . . , cn−n in order for
k
message digits to be transmitted rst, they must be
shifted right by
n−k
positions:
m(x) → (0, 0, . . . , 0, m0 , m1 , . . . , mk−1 ) =
xn−k m(x)
to generate valid codeword, low order n−k coe cients must be chosen to make complete
suppose that
n-tuple
a multiple of
xn−k m(x) has nonzero remainder when divided by g(x)
then a codeword (= multiple of the remainder Example:
g(x)
g(x))
Rg(x) (xn−k m(x))
31 mod 7 = 3
hence
can be formed by subtracting
from the shifted message
31 − 3 = 28
xn−k m(x)
is a multiple of 7.
Systematic generator matrix (2)
7-14
systematic encoder for cyclic code with generator polynomial
g(x)
acts as follows
m(x) 7→ xn−k m(x) − (xn−k m(x)
mod g(x))
=
xn−k m(x) − Rg(x) (xn−k m(x))
=
(0, . . . , 0, m0 , . . . , mk−1 ) − (p0 , . . . , pn−k−1 , 0, . . . , 0)
no monomials in common, thus the polynomial subtraction is a multiplexing operation:
105
this encoder creates a codeword by appending to the message digits the negative of the remainder modulo
g(x)
of the shifted message
polynomial
Systematic generator matrix (3)
7-15
systematic encoder uses remaindering to produce multiple of generator polynomial
remainder of division by xed polynomial is a linear function
⇒
systematic encoder is a linear transformation, so its output is
determined by output values for any basis
standard basis consists of the check symbols for message
−(xn−k · xi )
k
xi
are
mod g(x) = −xn−k+i
rows of the systematic generator matrix for
gi (x)
= −x
n−k+i
= −s
the
s[j] (x) := xj mod g(x)
n−k
coe cients of
[j]
i = 0, 1, . . . , k − 1
are
n−k+i
(x) + xn−k+i ,
(which is
s[j] (x)
mod g(x).
mod g(x) + x
[n−k+i]
where
{1, x, . . . , xk−1 }
unit vectors
Rg(x) (xj ))
are
[j]
[j]
(s0 , s1 , . . . , sn−k−1 )
Systematic generator matrix (4)
7-16
coe cients of s[j] (x) form the parity-check portion of the systematic generator matrix:
[n−k]
−s0 [n−k+1] −s0 G2 = . . . [n−1] −s0 rst row of
G2
is
[n−k]
... ...
−sn−k−1 [n−k+1] −sn−k−1
1 0
0 1
... ...
..
. . .
. . .
. . .
..
[n−1] −sn−k−1
0
0
...
.
...
−xn−k mod g(x) + xn−k ,
which by div.alg. equals
g(x): xn−k = 1 · g(x) + xn−k 106
.
0 0 . . . 1
mod g(x)
⇒
rst row of
G2
equals rst row of nonsystematic
systematic generator matrix has well
G1
≤ k(n − k + 1) nonzero elements (as
G1 )
both
G1
and
G2
are determined by
n−k
parity-check portion of each row of
G2
values (coe cients of
g(x))
comes from the rst row by
using a linear feedback shift register whose scalers are the coe cients of
g(x)
Linear feedback shift register Example: This LFSR multiplies by
x
7-17 modulo
x16 + x12 + x5 + 1:
Sage-notebook: LFSR
(see Sage-notebook for explanation)
Examples of binary cyclic codes (2)
7-18
systematic generator matrix for (4, 3) simple parity-check code:
g(x) = x + 1 ⇒ xj
in
F2 [x]
23 = 8
for
holds
1 mod g(x) = 1 ⇒ G = 1 1
1 0 0
0 1 0
0 0 . 1
x7 − 1 = (x + 1)(x3 + x + 1)(x3 + x2 + 1)
distinct divisors of
x7 − 1 ⇒
8 cyclic codes of blocklength 7
g(x) = x3 + x + 1, we obtain a cyclic Hamming code 1 1 0 1 0 0 0 1 0 0 1 0 1 0 1 1 0 1 0 0 ⇒ 0 1 0 1 1 1 G= 1 1 1 0 0 1 0 0 0 1 0 1 1 1 0 1 0 0 0 1
1 0 1
observe that the columns of this matrix are the coordinate representations of
xi
the dual code has generator matrix
H,
the (7, 3) simplex code.
(see third part of Sage notebook)
Cyclic Hamming codes
Theorem 7.5. For every
r≥2
there is a cyclic binary Hamming code.
Classification of binary cylic codes
107
Sage-notebook: Examples of cyclic codes 7-19
Sage-notebook: Cyclic Hamming code 7-20
recall some facts/consequences of the theorem on generator polynomial:
*
each cyclic code is uniquely identi ed by its generator polynomial
*
= unique monic polynomial dividing all codewords
*
if length is
n,
generator poly is a divisor of
xn − 1
consequence: we can nd all cyclic codes by listing all factors of xn −1
Sage-notebook: Classi cation cyclic codes
Examples of binary cyclic codes (3)
the cyclic polynomial
x15 − 1
=
there are
25
x15 − 1 ∈ F2 [x]
7-21 has ve distinct prime factors:
(x + 1)(x2 + x + 1)(x4 + x + 1)(x4 + x3 + 1)(x4 + x3 + x2 + x + 1) cyclic codes, examples:
4
(x + x + 1) (x4 + x + 1)(x4 + x3 + x2 + x + 1) (x4 + x + 1)(x4 + x3 + x2 + x + 1)(x2 + x + 1) (x4 + x + 1)(x4 + x3 + x2 + x + 1)(x2 + x + 1)(x + 1)
(15,11) binary cyclic Hamming (15,7) 2-error correcting BCH (15,5) 3EC BCH (15,4) simplex code
observe that these codes, with minimum distances 3, 5, 7, 8, are one after another obtained by expurgation (= remove codewords that do not satisfy additional restriction)
weight distributions of the example codes:
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
1
0
0
35
105
168
280
435
435
280
168
105
35
0
0
1
1
0
0
0
0
18
30
15
15
30
18
0
0
0
0
1
1
0
0
0
0
0
0
15
15
0
0
0
0
0
0
1
1
0
0
0
0
0
0
15
0
0
0
0
0
0
0
Cyclic codes: review
7-22
cyclic code = linear block code, s.t. every cyclic shift of a codeword is a codeword
a cyclic code has generator polynomial
g(x)
that is a divisor of every
codeword polynomial
the generator polynomial is a divisor of
xn − 1,
where
length
encoding: nonsystematic systematic
m(x) 7→ m(x)g(x)
m(x) 7→ xn−k m(x) − Rg(x) (xn−k m(x)) 108
n
is the block-
of
codewords can be characterized by:
c(x)
mod g(x)
=
0
7.3 Parity check polynomial and syndrome polynomial
Parity-check polynomial
7-23
parity-check polynomial of cyclic code with generator by
h(x) = degree =
g(x)
is de ned
xn − 1 g(x)
n − (n − k) = k
the parity-check polynomial de nes a relation satis ed by all codewords:
c(x)h(x)
=
m(x)g(x)h(x) = m(x)(xn − 1)
=
xn m(x) − m(x) = 0
in particular, coe cients of this corresponds to
xk , xk+1 , ..., xn−1
mod (xn − 1) c(x)h(x)
in
all vanish
n−k check equations (explaining the name check
polynomial):
x
xk
⇒ 0 = c0 hk + c1 hk−1 + . . . + ck−1 h1 + ck h0
k+1
⇒ 0 = c1 hk + c2 hk−1 + . . . + ck h1 + ck+1 h0
. . .
xn−1
. . .
⇒ 0 = cn−k−1 hk + cn−k hk−1 + . . . + cn−2 h1 + cn−1 h0
Parity-check matrix: nonsystematic
n−k
check equations obtained from
7-24
c(x)h(x) = 0 mod (xn − 1)
correspond to nonsystematic parity-check matrix:
hk 0 H1 = · · · 0 0
hk−1 hk
... hk−1
h1 ...
..
.
..
..
... ...
0 0
.
h0 h1
.
hk 0
109
hk−1 hk
0 h0
0 0
... ...
..
..
..
.
... hk−1
.
h1 ...
.
h0 h1
hR (x) 0 R 0 xh (x) . . . . = . . n−k−2 R 0 x h (x) h0 xn−k−1 hR (x)
Observations
hk 6= 0
h0 = 6 0 (else x would be a factor of xn − 1, so h(0) 6= 0
since
The rows of
deg h(x) = k
H1
(hence matrix is full-rank) since
h(x) divides xn − 1),
are shifts of the reverse polynomial of
h(x)
hR (x) = hk + hk−1 x + . . . + h1 xk−1 + h0 xk . Sage-command(s):
p.reverse()
Parity-check matrix: nonsystematic (2)
7-25
hR (x) = xk h(x−1 ), zeroes of hR (x) are reciprocals h(x), and hR (x) is called the reciprocal polynomial
since
following equation shows that
g R (x)hR (x)
parity-check matrix
hR (x)
divides
of zeroes of
xn − 1:
=
(g(x)h(x))R
=
(xn − 1)R = 1 − xn = −(xn − 1)
H1
has the form of a nonsystematic generator
matrix
rows of
H1
are shifts of
R h−1 0 h (x) erated by g(x)
thus
hR (x)
generates a cyclic code that is dual to the code gen-
cyclic code generated by
h(x) consists g(x)
of the reversals of the dual of
Sage-notebook: Parity check polynomial
the cyclic code generated by
Syndrome polynomial
7-26
two possibilities to measure a syndrome from the received word
r
1. multiply by nonsystematic parity check matrix:
rH1T = ⇒
last
coe cients of
r(x)h(x)
polynomial multiplication
2. remainder
⇒
n−k
r(x) mod g(x)
polynomial division
both can be very e ciently implemented by shift register circuits furthermore:
*
multiply-by-g(x)-circuit = nonsystematic encoder
*
divide-by-g(x)-circuit (plus multiplex) = systematic encoder
110
Syndrome circuit #1
7-27
syndrome computation circuit corresponding to plication by xed polynomial
input sequence
H1
performs multi-
h(x)
rn−1 , rn−2 , . . . , r0
(i.e.,
coe cients in falling or-
der) is convolved with parity-check polynomial coe cient sequence
h0 , h1 , . . . , hk output sequence consists of the coe cients of r(x)h(x) in falling order since
deg r(x) ≤ n − 1,
only
n−k
of the
the product
n+k
r(x)h(x)
coe cients of
drome: it consists of the coe cients of
these are generated after
has degree
≤n−1+k
r(x)h(x) are used as the synxk , . . . , xn−1 in r(x)h(x)
rn−1 , . . . , rn−k
have been shifted into the
register
Syndrome polynomial
7-28
we could obtain systematic parity-check matrix from systematic generator matrix using the general approach:
G = [P |I] ⇒ H = [I| − P T ] direct construction: de ne syndrome polynomial to be the remainder of division by generator polynomial:
s(x) = r(x)
mod g(x) = s0 + s1 x + . . . + sn−k−1 xn−k−1
every codeword is a multiple of
g(x),
so codewords have syndrome 0.
Thus
s(x)
= r(x)
mod g(x) = (c(x) + e(x))
= c(x)
mod g(x) + e(x)
mod g(x) = e(x)
remainder function is linear in the dividend therefore remainders of all
n-tuples mod g(x), (i = 0, 1, . . . , n − 1) 111
mod g(x) mod g(x)
r(x)
are linear combinations of
xi
Parity-check matrix: systematic
s(x)
The syndrome polynomial
7-29
corresponds to systematic parity-check
matrix:
T
1 x
. 1 0 . . n−k−1 0 1 x H2 = xn−k mod g(x) = .. .. . . . . 0 0 . xn−2 mod g(x) xn−1 mod g(x) i mod g(x).
Column
of
H2
is syndrome of
Special case: column coe cient 1 of
xn−k
Next: Columns
i+1
n−k
xi
[n−k]
[n−2]
... ...
0 0
s0 [n−k] s1
... ...
s0 [n−2] s1
..
.
. . .
. . .
..
. . .
...
1
sn−k−1
[n−k]
...
.
[n−2]
sn−k−1
and consists of coe cients of
[n−1] s0 [n−1] s1 . . . [n−1] sn−k−1
xi
−g(x) except leading 100 = −81 mod 181).
consists of coe cients of
(similar for numbers :
is obtained from column
i
by a linear feedback shift
(see below)
Syndrome circuit #2
7-30
Syndromes corresponding to
H2
can be calculated very e ciently using
linear feedback shift register circuits that implement polynomial division.
Encoding circuits can also be used for syndrome computation: syndrome = actual check symbols - expected check symbols where expected check symbols are computed from received message symbols.
Partial syndromes
7-31
The zeroes of the generator polynomial determine codewords:
c(x)
is a codeword if and only if
(The if holds when
g(x)
for every zero
β
of
g(x).
has no repeated zeroes, i.e., repeated factors.)
g(x) belong to some extension eld Fqm of Fq (see p. ??: FQ xQ − 1). Let {β1 , . . . , βt } include at least one zero of factor of g(x).
The zeroes of
consists of zeroes of each prime
c(β) = 0
112
The partial syndromes
S1 , . . . , St
r(x)
of
are
Si = r(βi ) = r0 + r1 βi + ... + rn−1 βin−1 The partial syndromes belong to the same extension eld as
β1 , . . . βt .
r0 , ..., rn−1 ∈ Fq , each partial syndrome Si de nes m linear equations Fq .
Since over
βi
The equations are linearly dependent if
is in a proper sub eld of
Fq m .
Example: binary cyclic Hamming code
p(x)
be a primitive polynomial over
7-32
F2
m
of degree
(i.e.,
p(x)
is
minimal polynomial of a primitive element).
equivalent: smallest
n
such that
cyclic code generated by
p(x)
has blocklength
systematic parity-check matrix m
H
= powers of
x ) has 2 − 1 ⇒ 1EC code
dromes of property
i
columns of
H= 1
p(x)|(xn − 1)
H
S1
n = 2m − 1.
xi mod p(x) = syncolumns ⇒ has Hamming
x = α ∈ F2m : α2
...
assume single error in place i, i.e., partial syndrome for
n = 2m − 1
(columns are
distinct nonzero
α
is
αn−2
αn−1
e(x) = xi
α:
= r(α) = r0 + r1 α + . . . + rn−1 αn−1 = c(α) + e(α) = e(α) = αi . i
decoder must nd error place base
from syndrome
S1 = αi
(logarithm
α ∈ F2m ).
Binary cyclic Hamming codes (cont.)
Fact: For any
m
7-33
there is a primitive polynomial
p(x)
over
F2
of degree
m. Follows from: For any
m
p0 + p1 x + ... + pm−1 xm−1
there is an irreducible polynomial
over
F2
of degree
m.
p(x) = p(x):
Some properties of
It is monic (leading coe cient 1)
F2m
x ∈ F2m
is isomorphic to is a zero of
F2 [x]/p(x)
p(x)
over
F2m :
p(x) = 0 but
mod p(x),
f (x) 6= 0 mod p(x) for any polynomial f (x) p(x) is a minimal polynomial for x
hence
113
of smaller degree,
Consequence Any binary Hamming code is equivalent to a cyclic Hamming-code. This is not true for nonbinary Hamming codes!
Nonbinary Hamming codes Every 1EC code has LI, hence no column There are
7-34
d∗ ≥ 3, hence any two columns of check of H is a multiple of another column.
matrix are
q m − 1 m-tuples
over Fq . The largest number of pairwise LI q m −1 columns is q−1 since we can use only one of the q − 1 nonzero multiples of any non-zero m-tuple. We normalize columns by requiring rst nonzero entry to be 1. Example:
1 0 0
1 0 1
1 0 2
1 1 0
1 1 1
1 1 2
1 2 0
1 2 1
1 2 2
0 1 0
0 1 1
0 0 1
0 1 2
Decoding procedure for this (13, 10) code:
compute syndrome
s = rH T .
Normalize syndrome by dividing by rst nonzero entry si . Equal column of
H
is error location, and
si
is error magnitude.
Cyclic nonbinary Hamming codes
7-35
A cyclic nonbinary Hamming code is de ned by an element order
n = (q m − 1)/(q − 1). H= 1
and
g(x)
H
β
β2
...
are LI over
Fq
β n−2 Fq
of
if and only if
β n−1
n
and
q−1
Fq m
of
β.
deg g(x) = m)
(Fact:
β j /β i = β `
Fact : There exists a cyclic Hamming code of blocklength if
of
The check matrix is
is the minimal polynomial over
The columns of
β
are coprime, which is true if and only if
m
is not in
n
Fq .
if and only
and
q−1
are
coprime. Example:
If
q=3
then
q − 1 = 2,
so odd values of
m
are required.
3
F33 be de ned by primitive polynomial x + 2x + 1, and β α = x (hence order is 13 as required) 1 0 0 1 2 0 2 0 H = 1 α2 . . . α22 α24 = 0 0 2 1 0 1 0 2 0 1 1 1 2 1 1 0
Let
The generator polynomial
= α2 , where 1 2 0
1 2 1
1 1 2
2 2 0
x3 +x2 +x+2 can be found by several methods,
then used to construct a systematic parity-check matrix.
114
1 2 2
Cyclic Golay code
7-36
Multiplicative orders of elements of There are
φ(23) = 22
F211
divide
211 − 1 = 2047 = 23 · 89.
elements of order 23. The conjugates of any such
β
are
β, β 2 , β 4 , β 8 , β 16 , β 9 , β 18 , β 13 , β 3 , β 6 , β 12 The minimal polynomial has degree 11. Prime polynomials of degree 11 are
g(x)
= x11 + x10 + x6 + x5 + x4 + x2 + 1
g˜(x)
= x11 + x9 + x7 + x6 + x5 + x + 1
These polynomials are mirror images; their zeroes are reciprocals.
β, β 2 , β 3 , β 4
The consecutive powers
5
among the conjugates guarantee
d∗ ≥
(this follows from the BCH bound, to be discussed later)
Lemma 7.6. Golay codewords of even weight have weight a multiple of 4. Hence the even weight subcode has minimum distance at least 8, and so:
Theorem 7.7. The cyclic Golay codes have
d∗ = 7,
dimension 12 and
are perfect. Proof. The distance statement follows by the same reasoning as for computing the minimum distance of an extended code (see p. 21). Dimension 12 is length 23 minus degree 11 of generating polynomial. Perfectness is then easily seen by verifying the sphere packing equation
23
2
12
=2
23 23 23 · 1+ + + 1 2 3
The weight enumerator is:
1 + 253x7 + 506x8 + 1288x11 + 1288x12 + 506x15 + 253x16 + x23
CRC-16 Cyclic codes are often used for error detection because the encoding and syndrome calculation circuits are very simple.
115
Sage-notebook: Golay23 as a cyclic code and the BCH bound 7-37
The most common generator polynomial is CRC-16:
x16 + x15 + x2 + 1 = (x + 1)(x15 + x + 1) The factor
15
2
− 1.
g1 (x) = x15 + x + 1
is primitive of degree 15 hence has order
Hence the code generated by
g1 (x)
[Recall: order of a polynomial = least integer
is a Hamming code.
n>0
such that
g(x)|(xn − 1).]
215 − 1 = 32767
Therefore the design blocklength of CRC-16 is
bits.
Almost always a signi cantly shortened code is used. Due to the factor
x+1, CRC-16 is the even-weight subcode of the Hamming Sage-notebook: CRC-16
code, hence minimum distance is 4.
CRC-CCITT
7-38
Another popular generator polynomial is CRC-CCITT:
x16 + x12 + x5 + 1 = (x + 1)p2 (x), where
p2 (x)
is a primitive polynomial of degree 15:
p2 (x) = x15 + x14 + x13 + x12 + x4 + x3 + x2 + x + 1. CRC-16 and CRC-CCITTT polynomials have only 4 nonzero coe cients, so the shift register coding circuits need only 3 exclusive-or gates.
Minimum distance for CRC-16, CRC-CCITT is 4.
Both codes correct
single errors while detecting double errors, or detect up to 3 errors. Will see in next chapter: Any cyclic code with
n − k = 16
detects burst
errors up to length 16 bits, which is optimal.
Syndromes of cyclic codes: review
For any check matrix
*
For
H
H,
the vector syndrome is
based on check polynomial
rH T ≡ (r(x)h(x) *
For systematic
H,
the
j -th
rH T ≡ r(x) Polynomial syndrome:
7-39
.
n
h(x) = (x − 1)/g(x),
mod (xn − 1)) ÷ xk
column is
xj mod g(x),
mod g(x) = Rg(x) (r(x))
s(x) = r(x) mod g(x).
116
s = rH T
so
Partial syndromes are de ned using the zeroes
β1 , . . . , β s
of
g(x):
Sj = r(βj ), j = 1, . . . , s The zeroes
Fqm
βj
and partial syndromes
of the channel alphabet
Sj
belong to an extension eld
Fq .
All the above syndromes contain the same information and can be obtained from each other by linear transformations. [The case where
g(x)
has multiple zeroes requires special consideration.]
117
118
7.4 Cyclic Hamming codes 7.5 BCH bound 7.6 Decoding BCH codes 7.7 Exercises 7.8 Examples and rst properties 8
Reed Solomon codes
8.1 Classic and historic view 8.2 Properties of MDS codes 8.3 Decoding RS-codes 8.4 PGZ decoder 8.5 Goppa codes 8.6 Berlekamp Massey decoder 8.7 Sugiyama decoder 8.8 Application: QR codes 8.9 Exercises 9
Building new codes from old ones, Part II
9.1 The (u, u + v)-construction 9.2 Concatenation and burst errors 9.3 Zyablov's trade-o and Justesen codes 9.4 Interleaved codes 9.5 Application: Data storage on CD and DVD 119
