信息论与编码理论基础 December 4, 2007

XDU, Fall 2007 Lecture Notes

Introduction to Channel Coding

I．Error control coding (overview)

„ Shannon showed that reliable communications can be achieved by proper coding of
information to be transmitted provided that the rate of information transmission is
below the channel capacity.
„ Coding is achieved by adding properly designed redundancy to each message before its
transmission. The added redundancy is used for error control. The redundancy may
appear in the form of extra symbols (or bits), or in the form of channel signal-set
expansion or in the form of combination of both.
„ Coding may be designed and performed separately from modulation, or designed in
conjunction with modulation as a single entity. In the former case, redundancy appears
in the form of extra symbols, normally called parity-check symbols.
„ Coding achieved by adding extra redundant digits is known as conventional coding, in
which error control (or coding gain) is achieved at the expense of bandwidth expansion
or data rate reduction. Therefore, conventional coding is suitable for error control in
power limited channels, such as deep space channel.
„ In the case that coding is designed in conjunction with modulation, redundancy comes
from channel signal-set expansion. This combination of coding and modulation is
usually known as coded modulation, which allows us to achieve error control (or
coding gain) without compromising bandwidth efficiency. We refer this technique as
the bandwidth efficient coding.
„ Historical notes:
Hamming codes (1950)
Reed-Muller codes (1954)
BCH codes (by Bose, Ray-Chaudhuri and Hocquenghem, 1959) ⎫
⎬ BM 算法（1968）
Reed-Solomon codes (1960) ⎭
Low-density parity-check codes (by Gallager in 1962, rediscovered in 90’s)
Convolutional codes (by Elias, 1955)
Viterbi algorithm (1967)
Concatenated codes (by Forney, 1966)
Trellis-coded modulation (by Ungerboeck, 1982)
Turbo codes (by Berrou , 1993)
Space-time codes (by Vahid Tarokh,1998)
„ Applications:
Deep space, satellite, mobile communications, voice modem, data networks, etc.
„ Two simple examples:

1
 0 → 000000 ⎫
Repetition codes: ⎬ (n, 1) code
1 → 111111 ⎭
Single parity-check codes:
0 0 0 0 0⎫
0 0 0 1 1⎪
⎪
⎬ (n, n-1) code
0 0 1 0 1⎪
0 0 1 1 0⎪
⎭
„ References:
[1] Shu Lin and D. J. Costello, Jr. Error Control Coding: Fundamentals and Applications.
2nd ed. Prentice-Hall, 2004.
[2] 王新梅，肖国镇．纠错码－原理与方法．西安：西安电子科技大学出版社，1991.
[3] D. J. Costello, J. Hagenauer, H. Imai, and S. B. Wicker, “Applications of error-control
coding,” IEEE Trans. Inform. Theory, vol.44, no.6, pp.2531-2560, Oct. 1998.
[4] D. J. Costello and G. D. Forney, “Channel coding: The road to channel capacity,”
Proceedings of The IEEE, vol.95, no.6, pp.1150-1177, June 2007.

II. Block coding

„ In block coding, information sequence is divided into messages of k information bits (or
symbols) each. Each message is mapped into a structured sequence of n bits (with n>k),
called a codeword.
(u0 , u1 , , uk −1 ) ↔ (c0 , c1 , , cn −1 )
message codeword

„ The mapping operation is called encoding. Each encoding operation is independent of

past encodings. The collection of all codewords is called an (n, k) block code, where n
and k are the length and dimension of the code, respectively.
„ In the process of encoding, n-k redundant bits are added to each message for protection
against transmission errors.
„ For example, consider a (5,2) binary code of size M=2k=4:
00 ↔ 10101 = c1
01 ↔ 10010 = c 2
C = {c1 , c 2 , c3 , c 4 }
10 ↔ 10010 = c3
11 ↔ 11110 = c 4
„ An important class of block codes is the class of linear block codes. A block code is said
to be linear if the vector sum of two codewords is also a codeword:
ci = (ci ,0 , ci ,1 , ci , n −1 ) ∈ C , c j = (c j ,0 , c j ,1 , c j , n −1 ) ∈ C

ci ⊕ c j = (ci ,0 ⊕ c j ,0 , ci ,1 ⊕ c j ,1 , , ci , n −1 ⊕ c j ,n −1 ) ∈ C

More general, a linear code is a subspace of GF( q ) n . (矢量加、标量乘运算封闭)

2
„ Linear block codes are normally put in systematic form:
(c0 , c1 , , cn −1 ) = ( c0 , c1 , , cn − k −1 , u0 , u1 , , uk −1 )
parity −check part message part

„ Each parity-check bit is a linear sum of message bits, i,e,

k −1
c j = ∑ pij ui , j = 0,1, , n − k − 1.
i =0

where pij =0 or 1. The n − k equations which gives the parity-check bits are called the

parity-check equations. They specify the encoding rule.

„ For an (n, k) block code, the ratios
k n−k
R= and η=
n n
are called code rate and redundancy, respectively.
„ An example for block code:
Let n=7 and k=4. Consider the (7, 4) linear systematic block code
Message: (u0 , u1 , u2 , u3 )

Codeword: (c0 , c1 , c2 , c3 , c4 , c5 , c6 ) = (c0 , c1 , c2 , u0 , u1 , u2 , u3 )

c0 = u0 + u1 + u2
Here, c1 = u1 + u2 + u3
c2 = u0 + u1 + u3
In matrix form:
⎡1 0 1 1 0 0 0⎤
⎢1 1 1 0 1 0 0 ⎥⎥
c = (u0 , u1 , u2 , u3 ) ⎢ = u ⋅G
⎢1 1 0 0 0 1 0⎥
⎢ ⎥
⎣0 1 1 0 0 0 1⎦

Encoder circuit:

3
 u

message register

To channel

⊕ ⊕
c0 c1
⊕
c2

III. Convolution Coding

„ During each unit of time, the input to convolutional is also a k-bit message block and the
corresponding is also an n-bit coded with k<n. (为避免混淆，可改为用k0, n0表示)
„ Each coded n-bit output block depends not only on the corresponding k-bit input
message block at the same time unit but also on the m previous message blocks.

u(1) c(1)
u(2) c(2)
u Encoder c
(memory)
u(k) c(n)

Input data stream

shift register

data frame
运算逻辑
有记忆

n n n n

encoder codeword frame

„ The code rate is defined as R = k / n .

The parameter m is called the memory order of the code.

4
„ In the process of coding, the information sequence u is divided into data frames of
length k. These subsequences of length k are applied to the k-input terminals of the
encoder, producing coded sequence of length n.
„ An example:
Let n=2, k=1 and m=2. Consider a rate-1/2 (2,1,2) convolutional code which is specified
by the following two generator sequences:
g (1) = (101), g (2) = (111)

c(1)

c
u

c(2)

Note: g (1) , g (2) 可看作编码器的两个冲激响应，由 u = δ = (100...) 得到。冲激响应至多

持续 m + 1 个时间单位，且可写为：
g (1) = ( g 0(1) , g1(1) ,… , g m(1) ) , g (2) = ( g 0(2) , g1(2) ,… , g m(2) )

- Let u = (u0 , u1 , ) be the input message sequence. Then the two output sequences are

c(1) = u * g (1) ⎫
⎬ 编码方程 (与冲激响应的卷积运算)
c(2) = u * g (2) ⎭

- At the lth time unit, the input is a single bit ul . The corresponding output is a block of two

bits, (cl(1) , cl(2) ) , which is given by

m
cl( j ) = ∑ ul −i gi( j ) = ul g 0( j ) + ul −1 g1( j ) + + ul − m g m( j )
i =1

⎧cl(1) = ul + ul − 2
⎪
⇒ ⎨cl2 = ul + ul −1 + ul − 2
⎪
⎩ memory

- The output codeword is given by c = ( c0(1) c0(2) , c1(1) c1(2) , ).

For u = (1011100 ), c = (11, 01, 00,10, 01,10,11, )

5
„ State Diagram: Since the encoder is a linear sequential circuit, its behavior can be
described by a state diagram. The encoder state at time l is represented by the message
bits stored in the memory units.
„ The encoder of the (2, 1, 2) convolutional code given in the example has 4 possible
states, and its state diagram is shown in the figure below .

0/00

00 input

0/11 1/11

output
1/00

01 10

0/01

1/10
0/10

11

1/01

label = input/output

„ Trellis diagram: The state diagram can be expanded in time to display the state
transition of a convolutional encoder in time. This expansion in time results in a trellis
diagram.

state
1/01
11 1/10 0/10
1/10
10 0/01
0/01
1/11 1/00
01
0/11
1/11 1/11
00 0/00 0/00 0/00 0/00

„ The encoding of a message sequence u is equivalent to tracing a path through the trellis.
„ The trellis structure is very useful in decoding a convolutional code.

6
IV. Conventional Coding
1. Types of codes
⎧ block codes - linear codes, cyclic codes ⎫
⎨ ⎬ classfication based on structure
⎩convolutional codes ⎭

⎧random-error-correcting codes
⎨
⎩burst-error-correcting codes

⎧ Binary codes
⎨
⎩ Nonbinary codes

⎧error-correction codes
⎨
⎩ error-detection codes
2. Error correcting capacity/ability
„ The error correcting capacity of a code C depends on its distance structure.

„ The Hamming distance between two codewords, x and y, in a code, denoted by

d H (x, y ) , is defined as the number of places in which they differ.

n
⎧1, if xi ≠ yi
d H (x, y ) = ∑ d H ( xi , yi ), d H ( xi , yi ) = ⎨
i =1 ⎩0, if xi = yi
or d H (x, y ) =|{i : xi ≠ yi }|

For example, d H (010,111) = 2, d H (30102, 21103) = 3

„ Hamming distance satisfies the axioms for a distance metric:

1) d H ( x, y ) ≥ 0, with equality iff x = y

2) d H (x, y ) = d H (y , x) (对称性)

3) d H (x, y ) ≤ d H (x, z ) + d H (z, y )

„ The minimum Hamming distance of a code C is defined as

d min min {d H (x, y ) | x, y ∈ C , x ≠ y}

„ For a convolutional code, this minimum Hamming distance is usually called the
minimum free distance, denoted by d free .

„ An (n, k) block code with minimum Hamming distance d min is capable of correcting

7
 ⎢ d − 1⎥
t = ⎢ min ⎥ or fewer random errors over a block of n digits (using minimum distance
⎣ 2 ⎦
decoding rule). This parameter t is called the error correcting capacity of the code.
d min
t t

decoding sphere d min ≥ 2t + 1

„ Error detection ability: e = d min − 1

„ Erasure correction ability: ρ = d min − 1

„ Up to ρ erasures and t random errors can be corrected if d min ≥ 2t + ρ + 1 .

„ The minimum Hamming distance of a linear block code depends on the choice of
parity-check equations and the number of parity bits, n − k .

3. Important codes
1) Algebraic block codes
- Hamming codes
- BCH codes: A large class of powerful multiple random error-correcting codes, rich in
algebraic structure, algebraic decoding algorithms available.
- Golay (23, 12) code: A perfect triple-error-correcting code, widely used and
generated by
g ( x) = 1 + x 2 + x 4 + x 5 + x 6 + x10 + x11
- Reed-Muller codes
- Reed-Solomon codes: nonbinary, correcting symbol errors or burst errors ,most widely
used for error control in data communications and data storages.
2) Convolutional codes: (2, 1, 6) code generated by
g (1) = (1101101), g (2) = (1001111)

This code has d free = 10 .

3) Codes (defined) on graphs:

Low-density parity-check codes ⎫
⎬ capacity-approching codes
Turbo codes ⎭
4. Types of error control schemes
- Forward-error-correction (FEC): An error-correction code is used.
- Automatic-repeat-request (ARQ): An error-detection code is used.

8
 If the presence of error is detected in a received word, a retransmission is requested. The
request signal is sent to the transmitter through a feedback channel. Retransmission
continues until no errors being detected.
- Hybrid ARQ: A proper combination of FEC and ARQ.

5. Decoding
„ Based on the received sequence, the encoding rules and the noise characteristics of the
channel, the receiver makes a decision which message was actually transmitted. This
decision making operation is called decoding.
„ Hard-decision
When binary coding is used, the modulator has only binary inputs (M=2). If binary
demodulator output quantization is used (Q=2), the decoder has only binary inputs. In this
case, the demodulator is said to make hard decision. Decoding based on hard decisions made
by the demodulator is called hard-decision decoding.
„ Soft-decision
If the output of demodulator consists of more than two quantization levels (Q>2) or is
left unquantized, the demodulator is said to make soft decisions. Decoding based on this is
called soft-decision decoding.

硬判决(Q=2)
Q 电平
解调器 译
量化器
码
软判决
器

„ Hard-decision decoding is much easier to implement than soft-decision decoding.

However, soft-decision decoding offers significant performance improvement over
hard-decision decoding. See figure 2.

9
 BPSK soft-decision bound
Shannon bound
1.2
BSC bound

1 soft-decision
Capacity (bits/symbol)

0.8

hard-decision
0.6

0.4

0.2

0
-2 0 2 4 6 8 10
E b/N0 (dB)

Figure 2 软判决与硬判决译码的信道容量
„ Optimal decoding
Given that y is received, the conditional error probability of decoding is defined as
P( E y ) P(cˆ ≠ c y )

Then the error probability of

P ( E ) = ∑ P ( E y ) P (y )
y

A decoding rule that minimizes P(E) is referred to as an optimal decoding rule.

Since minimize P(cˆ ≠ c y) is equivalent to maximize P(cˆ = c y) , we have

MAP rule: cˆ = arg max P(c y )

c

„ Maximum-likelihood decoding (MLD):

P(c) P(y c)
Note that P(c y ) = , we have
P(y )

ML rule: cˆ = arg max P ( y | c) (Suppose all the messages are equally likely)
c

6. MLD for a BSC

In coding for a BSC, every codeword and every received word are binary sequences.
„ Suppose some codeword is transmitted and the received word is y = ( y1 , y2 ,…, yn ) .

10
 For a codeword ci , the conditional probability P (y | ci ) is

P ( y | ci ) = p dH ( y ,ci ) (1 − p ) n − d H ( y ,ci )

For p<1/2, P (y | ci ) is a monotonially decreasing function of d H (y , ci ) . Then

P ( y | ci ) > P ( y | c j ) iff d H ( y , ci ) < d H ( y , c j )

„ MLD:
1) Compute d H (y , ci ) for all ci ∈ C .

2) ci is taken as the transmitted codeword if d H ( y , ci ) < d H ( y , c j ) for ∀j ≠ i .

3) Decoding ci into message ui .

This is called the minimum distance (nearest neighbor) decoding.

7. Performance measure and coding gain

„ Block-error probability: It is the probability that a decoded word is in error.
„ Bit-error probability: It is the probability that a decoded bit is in error.
„ The usual figure of merit for a communication system is the ratio of energy per
information bit to noise power spectral density, Eb / N 0 , that is required to achieve a

given error probability.

„ Coding gain of a coded communication system over an uncoded system with the same
modulation is defined the reduction, expressed in dB, in the required Eb/N0 to achieve a
target error probability.
⎡E ⎤ ⎡E ⎤
Coding gain = ⎢ b ⎥ −⎢ b⎥ (in dB)
⎣ N 0 ⎦ uncoded ⎣ N 0 ⎦ coded
„ Shannon limit: A theoretical limit on the minimum SNR required for coded system with
code rate Rc to achieve error-free information transmission. See figures 3 and 4.

11
Figure 3 Milestones in the drive towards channel capacity achieved over the past 50 years.

Figure 4 Performance of various coded modulation schemes.

12
V. Finite Fields (分组码的代数结构)
1. Binary arithmetic and field
„ Consider the binary set {0,1}, Define two binary operations, called addition ‘+’ and
multiplication ‘·’, on {0,1} as follows
+ 0 1 · 0 1
0 0 1 0 0 0
1 1 0 1 0 1

„ These two operations are commonly called module-2 addition and multiplication
respectively. They can be implemented with an XOR and an AND gate, respectively.
„ The set {0, 1} together with module-2 addition and multiplication is called a binary field,
denoted by GF(2) or F2 .

2. Vector space over GF(2)

„ A binary n-tuple is an ordered sequence, (a1,a2,…,an), with ai ∈ GF(2) .

- There are 2n distinct binary n-tuples.

- Define an addition operation for any two binary n-tuples as follows:
(a1 , a2 ,..., an ) + (b1 , b2 ,..., bn ) = (a1 + b1 , a2 + b2 ,..., an + bn )

where ai+bi is carried out in module-2 addition.

- Define a scalar multiplication between and element c in GF(2) and a binary n-tuple
(a1,a2,…,an) as follows:
c ⋅ (a1 , a2 ,..., an ) = (c ⋅ a1 , c ⋅ a2 ,..., c ⋅ an )

where c⋅ai is carried out in module-2 multiplication.

„ Let Vn denote the set of all 2n binary n-tuples. The set Vn together with the addition
defined for any two binary n-tuples in Vn and the scalar multiplication is called a vector
space over GF(2). The elements in Vn are called vectors.
- Note that Vn contains the all-zero n-tuple (0,0,…,0) and
(a1 , a2 ,..., an ) + (a1 , a2 ,..., an ) = (0, 0,..., 0)

„ Example: Let n=4. Then

⎧(0000), (0001), (0010), (0011), (0100), (0101), (0110), (0111) ⎫
V4 = ⎨ ⎬
⎩(1000), (1001), (1010), (1011), (1100), (1101), (1110), (1110) ⎭

„ A subset S of Vn is called a subspace of Vn if

1) the all-zero vector is in S;

13
 2) the sum of two vectors in S is also a vector in S；

For example: S={(0000),(0101),(1010),(1111)} forms a subspace of V4.

„ A linear combination of k vectors, V1 , V2 ,..., Vk in Vn is a vector of the form

V = c1V1 + c2 V2 + + ck Vk

where ci ∈ GF(2) and is called the coefficient of Vi.

„ The subspace formed by the 2k linear combinations of k linearly independent vectors

V1 , V2 ,..., Vk in Vn is called a k-dimensional subspace of Vn.

„ A binary polynomial is a polynomial with coefficients from the binary field.

For example, 1+x2, 1+x+x3.
- A binary polynomial p(x) of degree m is said to be irreducible if it is not divisible by
any binary polynomial of degree less than m and greater than zero. For example, 1+x+x2,
1+x+x3.
- An irreducible polynomial p(x) of degree m is said to be primitive if the smallest
positive integer n for which p(x) divides xn+1 is n=2m-1. For example, p(x)=1+x+x4. (it
divides x15+1)
- For any positive integer m, there exists a primitive polynomial of degree m. (可查表)
4. Galois fields
„ Groups: A group is an algebraic structure (G, *) consisting of a set G and an operation *
satisfying the following axioms:
1) Closure: For any a, b∈G, the element a*b is in G;
2) Associative law: For any a, b, c∈G, a*(b*c)=(a*b)*c;
3) Identity element: There is an element e∈G for which e*a=a*e=a for all a∈G;
4) Inverse: For every a∈G, there exists a unique element a-1∈G, such that a*a-1 =
a-1*a = e.
„ A group is called a commutative group or Abelian group if a*b = b*a for all a, b∈G.
Examples: - 整数，有理数，实数，with addition;
- Integers with module-m addition.
„ Fields: A field F is a set that has two operations defined on it : Addition and

multiplication, such that the following axioms are satisfied:

1) The set is an Abelian group under addition; (单位元称为‘0’)
2) The set is closed under multiplication, and the set {a∈F, a≠0} forms an Abelian

group (whose identity is called ‘1’) under the multiplication (*);

3) Distributive law: For all a, b, c∈F, (a+b)*c=(a*c)+(b*c).

我们经常用‘0’表示加法运算下的单位元，‘-a’表示a的加法逆。经常用‘1’表示乘法

14
运算下的单位元，‘a-1’表示a的乘法逆。这样，减法a-b means a+(-b), 除法a/b means b-1a.
„ A field with q elements, if it exists, is called a finite field, or a Galois field, and is
denoted by GF(q).
For example, GF (2) ——the smallest field

„ Let F be a field. A subset of F is called a subfield if it is a field under the inherited

addition and multiplication. 原来的域F称为 an extension field of the subfield.

„ In any field, if ab=ac and a≠0, then b=c.

„ For any positive integer m≥1,there exists a Galois field of 2m elements, denoted by
GF (2 m ) .

„ The construction of GF (2 m ) is very much the same as the construction of the

complex-number field from the real-number field.
We begin with a primitive polynomial p ( x ) of degree m with coefficients from the

binary field GF (2) .

- Let α be the root of p ( x ) ; i.e., p (α ) =0. Then the field elements can be represented

by {0,1, α , α 2 ,..., α q −2 } , where q=2m.

0 = α −∞ , 1 = α 0 , α 1 , α 2 , …, α q − 2 ,

1= α q −1 (since α is a root of p ( x ) and p( x) | x 2 −1

+ 1 ,α must be a root of x 2 −1
m
+1)
m

- For example: Construct GF (4) from GF (2) using p ( x) = x 2 + x + 1 .

Polynomial notation Binary notation Integer Exponential
0 00 0 0
1 01 1 1
x 10 2 α
α2
x+1 11 3
α3 =1=α0

„ The element α whose powers generate all the nonzero elements of GF (2 m ) is called

a primitive element of GF (2 m ) .

VI. Binary Linear Block codes

„ An (n, k) linear block code over GF ( 2) is simply a k-dim subspace of the vector

15
 space Vn of all the binary n-tuples.
„ In any linear code, the all-zero word, as the vector-space origion, is always a codeword
(∵if c is a codeword, then (-c) is also a codeword，so dose c+(-c)).
„ The Hamming weight w(c) of a vector c is the number of nonzero components of c.
Obviously, w(c) = d H (c, 0) .

„ The minimum Hamming weight of a code C is the smallest Hamming weight of any

nonzero codeword of C.

w m in = m in w H ( c )
c∈ C , c ≠ 0

For a linear code, d H (c1 , c 2 ) = d H (0, c 2 − c1 ) = d H (0, c) = w(c)

{ }
d min = min d H ( 0, ci − c j ) ci , c j ∈ C , i ≠ j = min w(c) = wmin
c≠0

1. Generator matrix
„ A generator matrix for a linear block code C of length n and dimension k is any k×n

matrix G whose rows form a basis for C.

Every codeword is a linear combination of the rows of G.

⎡ g 0 ⎤ ⎡ g 00 g 01 … g 0,n −1 ⎤
⎢g ⎥ ⎢ g g11 … g1,n −1 ⎥
⎥
G= ⎢ 1 ⎥
= ⎢ 10

⎢ ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥
⎣ g k −1 ⎦ ⎢⎣ g k −1,0 g k −1,1 … g k −1, n −1 ⎥⎦ k ×n

⎡g 0 ⎤
⎢g ⎥ k −1
„ Encoding procedure: c = u ⋅ G = [u0 , u1 , uk −1 ] ⎢ 1 ⎥ = ∑ ul g l
⎢ ⎥ l =0
⎢ ⎥
⎣g k −1 ⎦
Example: For a (6, 3) linear block code,
⎡g 0 ⎤ ⎡0 111 0 0 ⎤
G = ⎢⎢g1 ⎥⎥ = ⎢⎢1 0 1 0 1 0 ⎥⎥
⎢⎣g 2 ⎥⎦ ⎢⎣11 0 0 0 1⎥⎦

The codeword for the message u = (1 0 1) is

c = u ⋅ G = 1⋅ ( 0 111 0 0 ) + 0 ⋅ (1 0 1 01 0 ) + 1⋅ (11 0 0 0 1) = (1 0 11 0 1)

„ An (n, k) linear systematic code is completely specified by an k×n generator matrix of

the following form:

16
 ⎡g 0 ⎤ ⎡ p00 p01 p0,n − k −1 1 0 0⎤
⎢g ⎥ ⎢ p
⎢ 1 ⎥ ⎢ p11 p1,n − k −1 0 1 0 ⎥⎥
= [P Ik ]
10
G= =
⎢ ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥
⎣ g k −1 ⎦ ⎣⎢ pk −1,0 pk −1,1 pk −1,n − k −1 0 0 1⎦
P matrix with pij = 0 or1 k ×k identity matrix

For example, the (6, 3) code above is a systematic code:

c5 = u2
c4 = u1
c3 = u0

c2 = u0 + u1 ⎫
⎪
c1 = u0 + u2 ⎬ parity-check equations
c0 = u1 + u2 ⎪⎭
2. Parity-check matrix
„ An (n, k) linear code can also be specified by an (n-k)×n matrix H.
Let c = ( c0 c1 cn −1 ) be an n-tuple. Then c is a codeword iff

c ⋅ HT = 0 = ( 0 0 0)
n −k个

The matrix H is called a parity-check matrix. By definition,

GH T = 0

„ For an (n, k) linear systematic code with generator matrix G = [ P I k ] , the

parity-check matrix is

⎡ h0 ⎤
⎢ h ⎥
H = ⎢ 1 ⎥ = ⎡I
⎣ n−k − P
T
⎤⎦
⎢ ⎥
⎢ ⎥
⎣h n−k ⎦
„ Example: For (7, 4) Hamming code
⎡1101000 ⎤ ⎡ ⎤
⎢0110100⎥ ⎢100 1011 ⎥ ⎧c0 = u0 + u2 + u3
⎢ ⎥ ⎪
G = ⎢1110010 ⎥ ⇒ H 3×7 = ⎢010 1110⎥ ⇒ ⎨c1 = u0 + u1 + u2
⎢ ⎥
⎢ ⎥ ⎢ 001 0111⎥ ⎪c = u + u + u
⎢1010001 ⎥ ⎩ 2 1 2 3
⎣ P ⎦ ⎣⎢ ⎥
PT ⎦

{
Thus, block code C = c ∈ GF (q) n cHT = 0 . }

17
3. Syndrome decoding
„ Error vector (or error pattern): Let c be the transmitted codeword, and r be the received
word. Then the difference between r and c gives the pattern of errors: e = r - c (for

binary codes, e = r ⊕ c )
ej =1 indicates that the j-th position of r has an error.
„ Obviously, r = c + e .
„ There are in total 2n possible error patterns. Among them, only 2n-k patterns are
correctable by an (n, k) linear code.
„ To test whether a received vector r contains errors, we compute the following (n-k)-
tuple:
s = ( s0 , s1 , , sn − k −1 ) r ⋅ HT

= (c + e) ⋅ HT

= cH T + eH T = eH T
问题： s ⇒ e = ?
If s ≠ 0 ⇒ e ≠ 0
If s = 0 ⇒ 无错，e = 0 ； 或错误不可检： e ∈ C .
„ The (n-k)-tuple, s is called the syndrome of r.
Any method solving these n-k equations is a decoding method.
„ 最小距离译码就是找重量最轻的 e such that eHT = rHT = s
„ Syndrome decoding consists of these steps:
1) Calculate syndrome s = rH T of received n-tuple.

2) Find 最可能的错误图样 e with eH T = s ---> 非线性运算

3) 估计发送码字 ĉ = r - e .
4) Determine message û from the encoding equation cˆ = uG
ˆ .
„ Example: (7, 4) Hamming code. Suppose c = (1001011) is transmitted and r=(1001001)
is received. Then s = ( s0 , s1 , s2 ) = rH T = (111)

Let e = (e0 , e1 ,..., e6 ) be the error pattern. Since s = eH T , we have the following 3

equations :
1=e0+e3+e5+e6
1=e1+e3+e4+e5
1=e2+e4+e5+e6
There are 16 possible solutions, 其中 e=(0000010)是重量最小，是最可能发生的错误图
样，故 cˆ = r ⊕ e =(10010010)⊕(0000010)=(1001011).

18
„ Standard array

c1=0 c2 c3 … cM
e2 e2+c2 e2+c3 … e2+cM
e3 e3+c2 e3+c3 … e3+cM
… … … … …
…
e 2r e 2r + c 2 e 2r + c3 e 2r + c M

M = 2k , r = n-k
- Each row is called a coset.

4. Hamming codes
„ First class of codes devised for error correction.
„ For any positive integer m ≥ 3 , there exists a Hamming code with the following
parameters:
code length: n = 2m − 1

dimension: k = 2 m − m − 1
Number of parity-check symbols: n-k=m
Error correcting capability: t=1
Minimum distance: dmin=3

5. Hamming bound (sphere packing)

For a (n, k) linear block code over GF(q) with error correction ability t,
⎛ t ⎛n⎞ i⎞
n − k ≥ log q Vq (n, t ) = log q ⎜ ∑ ⎜ ⎟ ( q − 1) ⎟
⎝ i =0 ⎝ i ⎠ ⎠
t
⎛n⎞
其中 Vq (n, t ) = ∑ ⎜ ⎟ ( q − 1) 是半径为 t 的 Hamming 球的 volume.
i

i =0 ⎝ i ⎠

证明：在 GF(q)上的 n 维空间中，总共有 q n 个 n 维向量，有 M = q k 个码字（Hamming

球），因此有
q k ⋅Vq (n, t ) ≤ q n

从而
qn
≥ Vq (n, t )
qk

n − k ≥ log q Vq (n, t )

z Note: Hamming codes are one of few perfect codes – achieving Hamming bound.

19