Information Theory

Imre Csiszár is a Research Professor at the Rényi Institute of the Hungarian Academy of Sciences, where he has worked since 1961. He is also Professor Emeritus of the University of Technology and Economics, Budapest, a Fellow of the Institute of Electronic and Electrical Engineers (IEEE) and former President of the Hungarian Mathematical Society. He has received numerous awards, including the Shannon Award of the IEEE Information Theory Society (1996). János Körner is a Professor of Computer Science at the Sapienza University of Rome, where he has worked since 1992. Prior to this, he was a member of the Mathematical Institute of the Hungarian Academy of Sciences for over 20 years, and also worked at AT&T Bell Laboratories, Murray Hill, New Jersey, for two years.

Part I. Information Measures in Simple Coding Problems: 1. Source coding and hypothesis testing: information measures; 2. Types and typical sequences; 3. Some formal properties of Shannon's information measures; 4. Non-block source coding; 5. Blowing up lemma: a combinatorial digression; Part II. Two-Terminal Systems: 6. The noisy channel problem; 7. Rate-distortion trade-off in source coding and the source-channel transmission problem; 8. Computation of channel capacity and ∆-distortion rates; 9. A covering lemma: error exponent in source coding; 10. A packing lemma: on the error exponent in channel coding; 11. The compound channel revisited: zero-error information theory and extremal combinatorics; 12. Arbitrary varying channels; Part III. Multi-Terminal Systems: 13. Separate coding of correlated source; 14. Multiple-access channels; 15. Entropy and image size characteristics; 16. Source and channel networks; 17. Information-theoretic security.