Algebraic and stochastic coding theory

506 Pages·2012·3.911 MB·English

by  Dave K Kythe, Prem K Kythe

K13864_cover 1/30/12 3:58 PM Page 1
C M Y CM MY CY CMY K
ELECTRICAL ENGINEERING Kythe
•
Kythe
Algebraic Stochastic Coding Theory
and
Algebraic
Using a simple yet rigorous approach, Algebraic and Stochastic Coding Theory
makes the subject of coding theory easy to understand for readers with a thorough A
knowledge of digital arithmetic, Boolean and modern algebra, and probability
l
theory. It explains the underlying principles of coding theory and offers a clear, g Stochastic
detailed description of each code. More advanced readers will appreciate its e and
coverage of recent developments in coding theory and stochastic processes. b
r
a
After a brief review of coding history and Boolean algebra, the book introduces
linear codes, including Hamming and Golay codes. It then examines codes based ic Coding Theory

on the Galois field theory as well as their application in BCH and especially the a
n
Reed–Solomon codes that have been used for error correction of data transmissions
d
in space missions.
S
t
The major outlook in coding theory seems to be geared toward stochastic processes, o
and this book takes a bold step in this direction. As research focuses on error c
correction and recovery of erasures, the book discusses belief propagation and h
distributions. It examines the low-density parity-check and erasure codes that have a
opened up new approaches to improve wide-area network data transmission. s
t
It also describes modern codes, such as the Luby transform and Raptor codes, i
c
that are enabling new directions in high-speed transmission of very large data to
C
multiple users.
o
This robust, self-contained text fully explains coding problems, illustrating them with d
more than 200 examples. Combining theory and computational techniques, it will i
n
appeal not only to students but also to industry professionals, researchers, and
g
academics in areas such as coding theory and signal and image processing.

T
h
e
o
r
y
K13864
Dave K. Kythe | Prem K. Kythe
6000 Broken Sound Parkway, NW
Suite 300, Boca Raton, FL 33487
711 Third Avenue
an informa business New York, NY 10017
www.crcpress.com 2 Park Square, Milton Park www.crcpress.com
Abingdon, Oxon OX14 4RN, UK
Composite
Algebraic
Stochastic
and
Coding Theory
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2012 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20120405
International Standard Book Number-13: 978-1-4665-0562-9 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Algebraic
Stochastic
and
Coding Theory
Dave K. Kythe | Prem K. Kythe
Boca Raton London New York
CRC Press is an imprint of the
Taylor & Francis Group, an informa business
To Kiran, Bharti, Jay, Keshav, and Ida,
so near and dear to us.
TThhiiss ppaaggee iinntteennttiioonnaallllyy lleefftt bbllaannkk
Contents
Preface ............................................................. xiii
Notations and Abbreviations...................................... xix
1 Historical Background ............................................. 1
1.1 Codes Predating Hamming ....................................... 1
1.2 Codes Leading to ASCII ......................................... 2
1.2.1 Original Morse Code ........................................ 3
1.2.2 Baudot’s Code .............................................. 4
1.2.3 Murray’s Code .............................................. 5
1.2.4 Bacon’s Telegraph Code .................................... 5
1.2.5 Gauss and Weber’s Telegraph Code ......................... 6
1.2.6 ITA2........................................................ 6
1.2.7 US TTY .................................................... 6
1.2.8 ASCII-1967 Code............................................ 7
1.3 BCD Codes....................................................... 8
1.3.1 Four-Bit BCD Codes........................................ 8
1.3.2 Addition with 8421 and Excess-3 Codes..................... 10
1.3.3 Codes Larger than Four Bits................................ 13
1.3.4 Biquinary Code............................................ 14
2 Digital Arithmetic ................................................ 15
2.1 Number Systems ................................................ 15
2.1.1 Decimal Numbers .......................................... 15
2.1.2 Division Algorithm ........................................ 16
2.1.3 Base Conversion ........................................... 18
2.1.4 Conversion of Integers ..................................... 18
2.1.5 LCM and GCD ............................................ 19
2.1.6 Modulo .................................................... 19
2.2 Boolean and Bitwise Operations ................................. 19
2.2.1 Boolean Logical Operations ................................ 19
2.2.2 Bitwise Operations ........................................ 20
2.2.3 Applications ............................................... 21
2.2.4 XOR Swap Algorithm ..................................... 22
2.2.5 XOR Linked Lists ......................................... 23
2.2.6 Bit Shifts .................................................. 24
v
vi CONTENTS
2.2.7 Arithmetic Shifts .......................................... 24
2.2.8 Logical Shifts .............................................. 26
2.2.9 Circular Shifts ............................................. 26
2.2.10 Shift Registers ............................................ 26
2.2.11 Rotate through Carry .................................... 27
2.2.12 One’s and Two’s Complement Arithmetic ................. 28
2.3 Checksum ...................................................... 30
2.3.1 Fletcher’s Checksum ....................................... 32
2.3.2 Cyclic Redundancy Check (CRC) .......................... 33
2.4 Ring Counters .................................................. 35
2.5 Residues, Residue Classes, and Congruences ..................... 36
2.6 Integral Approximations ........................................ 40
2.7 Lexicographic Order ............................................ 40
3 Linear Codes ...................................................... 43
3.1 Linear Vector Spaces over Finite Fields .......................... 43
3.2 Communication Channels ....................................... 45
3.2.1 Shannon’s Theorems ....................................... 46
3.3 Some Useful Definitions ......................................... 48
3.3.1 Hamming Distance ........................................ 48
3.3.2 Lee Distance ............................................... 49
3.3.3 Hamming Cubes ........................................... 49
3.3.4 Hamming Weight .......................................... 50
3.3.5 Residual Code ............................................. 51
3.2.6 Tree Algorithm ............................................ 51
3.4 Linear Codes ................................................... 52
3.4.1 Linear Code ............................................... 52
3.4.2 Dual Codes ................................................ 52
3.4.3 Dimension of a Code ....................................... 52
3.4.4 Self-Dual Codes and Types ................................ 53
3.4.5 Advantages of Linear Codes ................................ 53
3.4.6 Popcount .................................................. 55
3.4.7 Parity Bits ................................................ 57
3.4.8 Uses of Parity Bits ......................................... 59
3.4.9 Stop Bit ................................................... 60
3.4.10 Parity-Check Matrix ...................................... 60
3.4.11 Decoding ................................................. 63
3.4.12 Decoding Methods ........................................ 64
3.4.12.1 Ideal Observer Decoding ............................. 64
3.4.12.2 Maximum Likelihood Decoding ...................... 64
3.4.12.3 Minimum Distance Decoding......................... 65
3.4.12.4 Nearest Neighbor Decoding .......................... 65
3.4.12.5 Standard Array Decoding ............................ 65
3.4.12.6 Syndrome Decoding ................................. 67
3.4.13 Syndrome Lookup Table .................................. 68
CONTENTS vii
3.5 Vector Operations .............................................. 69
3.5.1 Wedge Product ............................................ 70
3.5.2 Bar Product ............................................... 72
3.6 Sphere Packing ................................................. 72
4 Hamming Codes .................................................. 75
4.1 Error Correcting Codes ......................................... 76
4.1.1 Binary Linear Hamming Codes ............................. 76
4.2 Hamming (7,4) Code ............................................ 76
4.2.1 Encoding and Decoding .................................... 77
4.2.2 Computation of Parity Bits ................................ 78
4.2.3 Syndrome Method ......................................... 78
4.2.4 Parity Check Method ...................................... 80
4.2.5 Mapping Method .......................................... 81
4.2.6 Systematic Hamming codes ................................ 82
4.2.7 Codeword Sequences ....................................... 83
4.2.8 Multiple-Bit Errors ........................................ 84
4.3 Hamming (11,7) Code .......................................... 85
4.4 General Algorithm .............................................. 85
4.4.1 All Sixteen Possible Hamming (7,4) Codes ................. 88
4.4.2 Hamming Code for Longer Data ........................... 88
4.5 Hamming’s Original Algorithm .................................. 90
4.6 Equivalent Codes ............................................... 92
4.7 q-ary Hamming Codes .......................................... 93
5 Extended Hamming Codes ...................................... 95
5.1 SEC-DED Codes ................................................ 95
5.1.1 Extension of Hamming Codes .............................. 96
5.2 Hamming (8,4) Code ............................................ 97
5.3 Hamming (13,8) Code .......................................... 98
5.4 Hamming (32,26) Code ......................................... 99
5.5 Hamming (72,64) Code ........................................ 101
5.5.1 Modified Hamming (72,64) Code .......................... 103
5.5.2 Encoding of Hamming (72,64) Code ....................... 105
5.6 Hsiao Code .................................................... 106
5.6.1 Parity Encoder and Decoder .............................. 113
5.7 Product Notes ................................................. 115
5.8 Uses of Hamming Codes ....................................... 115
6 Bounds in Coding Theory ...................................... 117
6.1 Definitions ..................................................... 117
6.2 Sphere-Packing Bound ......................................... 119
6.3 Johnson Bound ................................................ 121
6.4 Gilbert–Varshamov Bound ..................................... 122
6.5 Hamming Bound .............................................. 123
6.6 Singleton Bound ............................................... 125
6.7 Plotkin Bound ................................................. 126
viii CONTENTS
6.8 Griesmer Bound ............................................... 127
6.9 Zyablov Bound ................................................ 128
6.10 Bounds in Fn ................................................ 129
2
6.11 Reiger Bound ................................................. 133
6.12 Krawtchouk Polynomials ..................................... 133
6.13 Linear Programming Bound .................................. 135
6.14 Stochastic Bounds for SEC-DED Codes ....................... 137
7 Golay Codes ..................................................... 143
7.1 Perfect Codes.................................................. 143
7.1.1 Repetition Code .......................................... 144
7.1.2 Hamming (7,4,3) Code .................................... 144
7.1.3 Binary Golay Codes ...................................... 144
7.1.4 Extended Binary Golay Codes ............................ 146
7.1.5 Ternary Golay Codes ..................................... 148
7.2 Geometrical Representation .................................... 149
7.2.1 Construction of Golay Codes.............................. 149
7.3 Other Construction Methods ................................... 152
7.4 Finite-State Codes ............................................. 154
7.5 MacWilliams’ Identity ......................................... 155
7.6 Golay’s Original Algorithm .................................... 156
7.7 Structure of Linear Codes ...................................... 158
7.7.1 Decoding of Linear Codes ................................. 160
8 Galois Fields ..................................................... 161
8.1 Finite Fields ................................................... 161
8.1.1 Extension Fields .......................................... 162
8.2 Construction of Galois Fields .................................. 165
8.3 Galois Fields of Order p ........................................ 167
8.4 Prime Fields ................................................... 169
8.5 Binary Fields .................................................. 169
8.5.1 Polynomial Basis Representation .......................... 170
8.6 Arithmetic in Galois Fields .................................... 171
8.6.1 Addition in Extension Field GF(2m) ...................... 172
8.6.2 Primitive Polynomials .................................... 172
8.6.3 Simple Test for Primitiveness ............................. 173
8.6.4 Gauss Elimination Method ................................ 174
8.7 Polynomials ................................................... 176
8.7.1 Division Algorithm ....................................... 176
8.7.2 Euclidean Algorithm ...................................... 177
8.7.3 Subfields ................................................. 179
8.8 Polynomial Codes ............................................. 180
9 Matrix Codes .................................................... 181
9.1 Matrix Group Codes ........................................... 181
9.2 Encoding and Decoding Matrices ............................... 182
9.3 Decoding Procedure ........................................... 182