Joint Source-Channel Coding

Andres Kwasinski, Rochester Institute of Technology, USA Dr. Kwasinski received his Ph.D. degree in Electrical and Computer Engineering from the University of Maryland in 2004. He is currently a Professor with the Department of Computer Engineering, Rochester Institute of Technology, Rochester, New York. Prior to this he was with Texas Instruments Inc., the Department of Electrical and Computer Engineering at the University of Maryland, and Lucent Technologies. Dr. Kwasinski has been a member of the IEEE Signal Processing Magazine Editorial Board, as Associate Editor and Area Editor for over twelve years. He was Editor for the IEEE Transactions on Wireless Communications and IEEE Wireless Communications Letters, the Globecom 2010 Workshop Co-Chair and the Chair of the IEEE Multimedia Technical Committee Interest Group on Distributed and Sensor Networks for Mobile Media Computing and Applications. He is a Senior Member of the IEEE. Vinay Chande, Qualcomm Inc., USA Vinay Chande has a Ph.D. in Electrical Engineering from the University of Maryland and his engineering education from Indian Institute of Technology, Mumbai. Dr. Chande works as a Systems Engineer at Wireless Research and Development at Qualcomm Technologies Inc. His current work gives him an opportunity to participate in and witness the advances in millimeter-wave radio bands, unlicensed spectrum access and machine learning for Industrial IoT.

Preface xi

1 Introduction and Background 1

1.1 Simplified Model for a Communication System 2

1.2 Entropy and Information 3

1.3 Introduction to Source Coding 6

1.3.1 Sampling and Quantization of Signals 6

1.3.2 Source Coding of Quantized Signals 9

1.3.3 Distortion and Rate-distortion Theory 13

1.4 Channels, Channel Coding, and Capacity 17

1.4.1 Channel Models 17

1.4.2 Wireless Channels 19

1.4.3 Channel Coding and Channel Capacity 23

1.5 Layered Model for a Communication System 26

1.6 Distortion, Quality of Service, and Quality of Experience 30

1.6.1 Objective Measurements of Distortion or Quality 31

1.6.2 Subjective and Perceptually Based Measurements of Distortion or Quality 32

1.7 Shannon’s Separation Principle and Joint Source–Channel Coding 36

1.8 Major Classes of Joint Source–Channel Coding Techniques 40

References 42

2 Source Coding and Signal Compression 43

2.1 Types of Sources 43

2.2 Lossless Compression 46

2.2.1 Entropy Coding 47

2.2.2 Predictive Coding 52

2.3 Lossy Compression 54

2.3.1 Quantization 54

2.3.2 Differential Coding 62

2.3.3 Transform Coding 63

2.3.4 Subband and Wavelet Coding 65

2.4 Embedded and Layered Coding 68

2.5 Coding of Practical Sources 71

2.5.1 Image Coding - JPEG 71

2.5.2 Embedded Image Coding – SPIHT 75

2.5.3 Video Coding 78

2.5.4 Speech Coding 83

References 86

3 Channel Coding 87

3.1 Linear Block Codes 87

3.1.1 Binary Linear Block Codes 90

3.1.2 Generator Matrix, Parity-Check Matrix, and Syndrome Testing 91

3.1.3 Common Linear Block Codes 92

3.1.4 Error and Erasure Correction with Block Codes 95

3.2 Convolutional Codes 97

3.2.1 Code Characterization: State and Trellis Diagrams 98

3.2.2 Maximum Likelihood (ML) Decoding 100

3.2.3 The Viterbi Algorithm 101

3.2.4 Error Correction Performance 104

3.3 Modified Linear Codes (Puncturing, Shortening, Expurgating, Extending, Augmenting, and Lengthening) 105

3.4 Rate-Compatible Channel Codes 105

References 110

4 Concatenated Joint Source–Channel Coding 111

4.1 Concatenated JSCC Bit Rate Allocation 111

4.2 Performance Characterization 119

4.2.1 Practical Source and Channel Codecs 119

4.3 Application Cases 131

References 133

5 Unequal Error Protection Source–Channel Coding 135

5.1 Effect of Channel Errors on Source Encoded Data 135

5.2 Priority Encoding Transmission Schemes for Unequal Loss Protection 142

5.3 Dynamic Programming Algorithm for Optimal UEP 147

5.4 Unequal Error Protection Using Digital Fountain Codes 163

References 171

6 Source–Channel Coding with Feedback 173

6.1 Joint Source–Channel Coding Formulation for a System with ACK/NACK Feedback 173

6.1.1 Performance Measurement 175

6.1.2 Classification of the Transmitters 176

6.1.3 Decoder Structure and Design 177

6.2 Packet Combining for Joint Source–Channel ARQ over Memoryless Channels 179

6.2.1 Decoder Design Problem 179

6.3 Pruned Tree-Structured Quantization in Noise and Feedback 193

6.3.1 Pruned Tree-Structured Vector Quantizers 194

6.3.2 Progressive Transmission with ACK/NACK Feedback of TSVQ-Encoded Sources 195

6.3.3 Progressive Transmission and Receiver-Driven Rate Control 204

6.4 Delay-Constrained JSCC Using Incremental Redundancy with Feedback 205

6.4.1 System Description 205

6.4.2 Optimal Source and Channel Rate Allocations Design 208

6.4.3 Performance 213

References 220

7 Quantizers Designed for Noisy Channels 223

7.1 Channel-Optimized Quantizers 223

7.2 Scalar Quantizer Design 227

7.3 Vector Quantizer Design 234

7.4 Channel Mismatch Considerations 245

7.5 Structured Vector Quantizers 249

References 255

8 Error-Resilient Source Coding 257

8.1 Multiple-Description Coding 257

8.2 Error-Resilient Coded Bit Streams 273

8.2.1 Robust Entropy Coding 273

8.2.2 Predictive Coding Mode Selection 279

References 281

9 Analog and Hybrid Digital–Analog JSCC Techniques 283

9.1 Analog Joint Source–Channel Coding Techniques 283

9.1.1 Analog Joint Source–Channel Coding in Vector Spaces 283

9.1.2 Analog Joint Source–Channel Coding Through Artificial Neural Networks 293

9.2 Hybrid Digital–Analog JSCC Techniques 297

References 302

10 Joint Source–Channel Decoding 305

10.1 Source-Controlled Channel Decoding 305

10.2 Exploiting Residual Redundancy at the Decoder 314

10.2.1 The Soft Output Viterbi Algorithm (SOVA) 315

10.2.2 Exploiting Residual Redundancy to Estimate A Priori Information 318

10.3 Iterative Source–Channel Decoding 323

10.3.1 The Channel Coding Optimal Estimation Algorithm 328

10.3.2 Channel Coding Optimal Estimation Applied to JSCD 330

References 333

11 Recent Applications and Emerging Designs in Source–Channel Coding 335

11.1 Source–Channel Coding for Wireless Sensor Networks 335

11.2 Extending Network Capacity Through JSCC 343

11.2.1 Video Telephony Calls as Application Example 345

11.2.2 CDMA Statistical Multiplexing Resource Allocation and Flow Control 347

11.2.3 Overhead from Communicating Rate-Distortion Data 354

11.2.4 Analysis for Dynamic Call Traffic and Admission Control 356

11.2.5 Performance Results 358

11.3 Source–Channel Coding and Cognitive Radios 364

11.4 Design of JSCC Schemes Based on Artificial Neural Networks 374

References 378

Index 381