Elements of Information Theory

About the authors THOMAS M. COVER is Professor jointly in the Departments of Electrical Engineering and Statistics at Stanford University. He is past President of the IEEE Information Theory Society and is a Fellow of the Institute for Mathematical Statistics and of the IEEE. In 1972 he received the Outstanding Paper Award in information Theory for his paper "Broadcast Channels," and he was selected in 1990 as the Shannon Lecturer, regarded as the highest honor in information theory. Author of over 90 technical papers, he is coeditor of the book Open Problems in Communication and Computation. Professor Cover has devoted the last 20 years to developing the relationship between information theory and statistics. He received his PhD in electrical engineering from Stanford University. JOY A. THOMAS is working at the IBM T. J. Watson Research Center in Hawthorne, New York. He received the IEEE Charles LeGeyt Fortescue Fellowship for 1984 85 and the IBM Graduate Fellowship for 1987 90. He received his BTech in electrical engineering at the Indian Institute of Technology Madras, India and his PhD in electrical engineering at Stanford University.

Entropy, Relative Entropy and Mutual Information. The Asymptotic Equipartition Property. Entropy Rates of a Stochastic Process. Data Compression. Gambling and Data Compression. Kolmogorov Complexity. Channel Capacity. Differential Entropy. The Gaussian Channel. Maximum Entropy and Spectral Estimation. Information Theory and Statistics. Rate Distortion Theory. Network Information Theory. Information Theory and the Stock Market. Inequalities in Information Theory. Bibliography. List of Symbols. Index.