Video Processing and Communications

Yao Wang received the B.S. and M.S. degrees in electrical engineering from Tsinghua University, Beijing, China, in 1983 and 1985, respectively, and the Ph.D. degree in electrical and computer engineering from the University of California at Santa Barbara in 1990. Since 1990, she has been with the Faculty of Electrical Engineering, Polytechnic University, Brooklyn, NY. Her research areas include video communications, multimedia signal processing, and medical imaging. She has authored and co-authored over 100 papers in journals and conference proceedings. She is a senior member of IEEE and has served as an Associate Editor for the IEEE Transactions on Circuits and Systems for Video Technology and the IEEE Transactions on Multimedia. She won the Mayor's Award of the City of New York for Excellence in Science and Technology in the Young Investigator category in 2000. Jorn Ostermann studied electrical engineering and communications engineering at the University of Hannover and Imperial College London, respectively. He received Dipl.-Ing. and Dr.-Ing. from the University of Hannover in 1988 and 1994, respectively. He has been a staff member with Image Processing and Technology Research, AT&T Labs Research since 1996, where he is engaged in research on video coding, shape coding, multi-modal human-computer interfaces with talking avatars, standardization, and image analysis. He is a German National Foundation scholar. In 1998, he received the AT&T Standards Recognition Award and the ISO award. He is a member of the IEEE, the IEEE Technical Committee on Multimedia Signal Processing, and chair of the IEEE CAS Visual Signal Processing and Communications (VSPC) Technical Committee. Ya-Qin Zhang received the B.S. and M.S. degrees in electrical engineering from the University of Science and Technology of China (USTC) in 1983 and 1985, respectively, and the Ph.D. degree from George Washington University in 1989. He is currently the Managing Director of Microsoft Research in Beijing, after leaving his post as the Director of the Multimedia Technology Laboratory at the Sarnoff Corporation in Princeton, NJ (formerly the David Sarnoff Research Center, and RCA Laboratories). He has been engaged in research and commercialization of MPEG2/DTV, MPEG4/VLBR, and multimedia information technologies. He has authored and co-authored over 200 refereed papers in leading international conference proceedings and journals. He has been granted over 40 U.S. patents in digital video, Internet, multimedia, wireless and satellite communications. He was the Editor-in-Chief of the IEEE Transactions on Circuits and Systems for Video Technology from 1997 to 1999. He is a Fellow of the IEEE.

(NOTE: Each chapter concludes with Summary, Problems, and Bibliography.

 1. Video Formation, Perception, and Representation.


Color Perception and Specification. Video Capture and Display. Analog Video Raster. Analog Color Television Systems. Digital Video.



 2. Fourier Analysis of Video Signals and Frequency Response of the Human Visual System.


Multidimensional Continuous-Space Signals and Systems. Multidimensional Discrete-Space Signals and Systems. Frequency Domain Characterization of Video Signals. Frequency Response of the Human Visual System.



 3. Video Sampling.


Basics of the Lattice Theory. Sampling over Lattices. Sampling of Video Signals. Filtering Operations in Cameras and Display Devices.



 4. Video Sampling Rate Conversion.


Conversion of Signals Sampled on Different Lattices. Sampling Rate Conversion of Video Signals.



 5. Video Modeling.


Camera Model. Illumination Model. Object Model. Scene Model. Two-Dimensional Motion Models.



 6. Two-Dimensional Motion Estimation.


Optical Flow. General Methodologies. Pixel-Based Motion Estimation. Block-Matching Algorithm. Deformable Block-Matching Algorithms. Mesh-Based Motion Estimation. Global Motion Estimation. Region-Based Motion Estimation. Multiresolution Motion Estimation. Application of Motion Estimation in Video Coding.



 7. Three-Dimensional Motion Estimation.


Feature-Based Motion Estimation. Direct Motion Estimation. Iterative Motion Estimation.



 8. Foundations of Video Coding.


Overview of Coding Systems. Basic Notions in Probability and Information Theory. Information Theory for Source Coding. Binary Encoding. Scalar Quantization. Vector Quantization.



 9. Waveform-Based Video Coding.


Block-Based Transform Coding. Predictive Coding. Video Coding Using Temporal Prediction and Transform Coding.



10. Content-Dependent Video Coding.


Two-Dimensional Shape Coding. Texture Coding for Arbitrarily Shaped Regions. Joint Shape and Texture Coding. Region-Based Video Coding. Object-Based Video Coding. Knowledge-Based Video Coding. Semantic Video Coding. Layered Coding System.



11. Scalable Video Coding.


Basic Modes of Scalability. Object-Based Scalability. Wavelet-Transform-Based Coding.



12. Stereo and Multiview Sequence Processing.


Depth Perception. Stereo Imaging Principle. Disparity Estimation. Intermediate View Synthesis. Stereo Sequence Coding.



13. Video Compression Standards.


Standardization. Video Telephony with H.261 and H.263. Standards for Visual Communication Systems. Consumer Video Communications with MPEG-1. Digital TV with MPEG-2. Coding of Audiovisual Objects with MPEG-4. Video Bit Stream Syntax. Multimedia Content Description Using MPEG-7.



14. Error Control in Video Communications.


Motivation and Overview of Approaches. Typical Video Applications and Communications Networks. Transport-Level Error Control. Error-Resilient Encoding. Decoder Error Concealment. Encoder-Decoder Interactive Error Control. Error-Resilience Tools in H.263 and MPEG-4.



15. Streaming Video over the Internet and Wireless IP Networks.


Architecture for Video Streaming Systems. Video Compression. Application-Layer QoS Control for Streaming Video. Continuous Media Distribution Services. Streaming Servers. Media Synchronization. Protocols for Streaming Video. Streaming Video over Wireless IP Networks.



Appendix A. Determination of Spatial-Temporal Gradients.


First- and Second-Order Gradient. Sobel Operator. Difference of Gaussian Filters.



Appendix B. Gradient Descent Methods.


First-Order Gradient Descent Method. Steepest Descent Method. Newton's Method. Newton-Ralphson Method. Bibliography.



Appendix C. Glossary of Acronyms.


Appendix D. Answers to Selected Problems.