Practical Image and Video Processing Using MATLAB

Oge Marques, PhD, is Associate Professor in the Department of Computer & Electrical Engineering and Computer Science at Florida Atlantic University. He has been teaching and doing research on image and video processing for more than twenty years, in seven different countries. Dr. Marques is the coauthor of Processamento Digital de Imagens and Content-Based Image and Video Retrieval and was editor-in-chief of the Handbook of Video Databases, a comprehensive work with contributions from more than 100 world experts in the field. He is a Senior Member of both the IEEE and the ACM.

LIST OF FIGURES xxi LIST OF TABLES xxxix FOREWORD xli PREFACE xliii ACKNOWLEDGMENTS xlix PART I IMAGE PROCESSING 1 INTRODUCTION AND OVERVIEW 3 1.1 Motivation / 3 1.2 Basic Concepts and Terminology / 5 1.3 Examples of Typical Image Processing Operations / 6 1.4 Components of a Digital Image Processing System / 10 1.5 Machine Vision Systems / 12 1.6 Resources / 14 1.7 Problems / 18 2 IMAGE PROCESSING BASICS 21 2.1 Digital Image Representation / 21 2.1.1 Binary (1-Bit) Images / 23 2.1.2 Gray-Level (8-Bit) Images / 24 2.1.3 Color Images / 25 2.1.4 Compression / 26 2.2 Image File Formats / 27 2.3 Basic Terminology / 28 2.4 Overview of Image Processing Operations / 30 2.4.1 Global (Point) Operations / 31 2.4.2 Neighborhood-Oriented Operations / 31 2.4.3 Operations Combining Multiple Images / 32 2.4.4 Operations in a Transform Domain / 32 3 MATLAB BASICS 35 3.1 Introduction to MATLAB / 35 3.2 Basic Elements of MATLAB / 36 3.2.1 Working Environment / 36 3.2.2 Data Types / 37 3.2.3 Array and Matrix Indexing in MATLAB / 37 3.2.4 Standard Arrays / 37 3.2.5 Command-Line Operations / 38 3.3 Programming Tools: Scripts and Functions / 38 3.3.1 M-Files / 39 3.3.2 Operators / 40 3.3.3 Important Variables and Constants / 42 3.3.4 Number Representation / 42 3.3.5 Flow Control / 43 3.3.6 Code Optimization / 43 3.3.7 Input and Output / 43 3.4 Graphics and Visualization / 43 3.5 Tutorial 3.1: MATLAB-a Guided Tour / 44 3.6 Tutorial 3.2: MATLAB Data Structures / 46 3.7 Tutorial 3.3: Programming in MATLAB / 53 3.8 Problems / 59 4 THE IMAGE PROCESSING TOOLBOX AT A GLANCE 61 4.1 The Image Processing Toolbox: an Overview / 61 4.2 Essential Functions and Features / 62 4.2.1 Displaying Information About an Image File / 62 4.2.2 Reading an Image File / 64 4.2.3 Data Classes and Data Conversions / 65 4.2.4 Displaying the Contents of an Image / 68 4.2.5 Exploring the Contents of an Image / 69 4.2.6 Writing the Resulting Image onto a File / 70 4.3 Tutorial 4.1: MATLAB Image Processing Toolbox-a Guided Tour / 72 4.4 Tutorial 4.2: Basic Image Manipulation / 74 4.5 Problems / 80 5 IMAGE SENSING AND ACQUISITION 83 5.1 Introduction / 83 5.2 Light, Color, and Electromagnetic Spectrum / 84 5.2.1 Light and Electromagnetic Spectrum / 84 5.2.2 Types of Images / 85 5.2.3 Light and Color Perception / 86 5.2.4 Color Encoding and Representation / 87 5.3 Image Acquisition / 89 5.3.1 Image Sensors / 89 5.3.2 Camera Optics / 92 5.4 Image Digitization / 93 5.4.1 Sampling / 95 5.4.2 Quantization / 96 5.4.3 Spatial and Gray-Level Resolution / 97 5.5 Problems / 101 6 ARITHMETIC AND LOGIC OPERATIONS 103 6.1 Arithmetic Operations: Fundamentals and Applications / 103 6.1.1 Addition / 104 6.1.2 Subtraction / 106 6.1.3 Multiplication and Division / 109 6.1.4 Combining Several Arithmetic Operations / 110 6.2 Logic Operations: Fundamentals and Applications / 111 6.3 Tutorial 6.1: Arithmetic Operations / 113 6.4 Tutorial 6.2: Logic Operations and Region of Interest Processing / 118 6.5 Problems / 122 7 GEOMETRIC OPERATIONS 125 7.1 Introduction / 125 7.2 Mapping and Affine Transformations / 127 7.3 Interpolation Methods / 130 7.3.1 The Need for Interpolation / 130 7.3.2 A Simple Approach to Interpolation / 131 7.3.3 Zero-Order (Nearest-Neighbor) Interpolation / 132 7.3.4 First-Order (Bilinear) Interpolation / 132 7.3.5 Higher Order Interpolations / 132 7.4 Geometric Operations Using MATLAB / 132 7.4.1 Zooming, Shrinking, and Resizing / 133 7.4.2 Translation / 134 7.4.3 Rotation / 134 7.4.4 Cropping / 134 7.4.5 Flipping / 134 7.5 Other Geometric Operations and Applications / 134 7.5.1 Warping / 134 7.5.2 Nonlinear Image Transformations / 135 7.5.3 Morphing / 137 7.5.4 Seam Carving / 137 7.5.5 Image Registration / 137 7.6 Tutorial 7.1: Image Cropping, Resizing, Flipping, and Rotation / 138 7.7 Tutorial 7.2: Spatial Transformations and Image Registration / 142 7.8 Problems / 149 8 GRAY-LEVEL TRANSFORMATIONS 151 8.1 Introduction / 151 8.2 Overview of Gray-level (Point) Transformations / 152 8.3 Examples of Point Transformations / 155 8.3.1 Contrast Manipulation / 155 8.3.2 Negative / 157 8.3.3 Power Law (Gamma) Transformations / 157 8.3.4 Log Transformations / 159 8.3.5 Piecewise Linear Transformations / 160 8.4 Specifying the Transformation Function / 161 8.5 Tutorial 8.1: Gray-level Transformations / 163 8.6 Problems / 169 9 HISTOGRAM PROCESSING 171 9.1 Image Histogram: Definition and Example / 171 9.2 Computing Image Histograms / 173 9.3 Interpreting Image Histograms / 174 9.4 Histogram Equalization / 176 9.5 Direct Histogram Specification / 181 9.6 Other Histogram Modification Techniques / 184 9.6.1 Histogram Sliding / 185 9.6.2 Histogram Stretching / 185 9.6.3 Histogram Shrinking / 186 9.7 Tutorial 9.1: Image Histograms / 188 9.8 Tutorial 9.2: Histogram Equalization and Specification / 191 9.9 Tutorial 9.3: Other Histogram Modification Techniques / 195 9.10 Problems / 200 10 NEIGHBORHOOD PROCESSING 203 10.1 Neighborhood Processing / 203 10.2 Convolution and Correlation / 204 10.2.1 Convolution in the One-Dimensional Domain / 204 10.2.2 Convolution in the Two-Dimensional Domain / 206 10.2.3 Correlation / 208 10.2.4 Dealing with Image Borders / 210 10.3 Image Smoothing (Low-pass Filters) / 211 10.3.1 Mean Filter / 213 10.3.2 Variations / 213 10.3.3 Gaussian Blur Filter / 215 10.3.4 Median and Other Nonlinear Filters / 216 10.4 Image Sharpening (High-pass Filters) / 218 10.4.1 The Laplacian / 219 10.4.2 Composite Laplacian Mask / 220 10.4.3 Directional Difference Filters / 220 10.4.4 Unsharp Masking / 221 10.4.5 High-Boost Filtering / 221 10.5 Region of Interest Processing / 222 10.6 Combining Spatial Enhancement Methods / 223 10.7 Tutorial 10.1: Convolution and Correlation / 223 10.8 Tutorial 10.2: Smoothing Filters in the Spatial Domain / 225 10.9 Tutorial 10.3: Sharpening Filters in the Spatial Domain / 228 10.10 Problems / 234 11 FREQUENCY-DOMAIN FILTERING 235 11.1 Introduction / 235 11.2 Fourier Transform: the Mathematical Foundation / 237 11.2.1 Basic Concepts / 237 11.2.2 The 2D Discrete Fourier Transform: Mathematical Formulation / 239 11.2.3 Summary of Properties of the Fourier Transform / 241 11.2.4 Other Mathematical Transforms / 242 11.3 Low-pass Filtering / 243 11.3.1 Ideal LPF / 244 11.3.2 Gaussian LPF / 246 11.3.3 Butterworth LPF / 246 11.4 High-pass Filtering / 248 11.4.1 Ideal HPF / 248 11.4.2 Gaussian HPF / 250 11.4.3 Butterworth HPF / 250 11.4.4 High-Frequency Emphasis / 251 11.5 Tutorial 11.1: 2D Fourier Transform / 252 11.6 Tutorial 11.2: Low-pass Filters in the Frequency Domain / 254 11.7 Tutorial 11.3: High-pass Filters in the Frequency Domain / 258 11.8 Problems / 264 12 IMAGE RESTORATION 265 12.1 Modeling of the Image Degradation and Restoration Problem / 265 12.2 Noise and Noise Models / 266 12.2.1 Selected Noise Probability Density Functions / 267 12.2.2 Noise Estimation / 269 12.3 Noise Reduction Using Spatial-domain Techniques / 269 12.3.1 Mean Filters / 273 12.3.2 Order Statistic Filters / 275 12.3.3 Adaptive Filters / 278 12.4 Noise Reduction Using Frequency-domain Techniques / 278 12.4.1 Periodic Noise / 279 12.4.2 Bandreject Filter / 280 12.4.3 Bandpass Filter / 281 12.4.4 Notch Filter / 282 12.5 Image Deblurring Techniques / 283 12.5.1 Wiener Filtering / 286 12.6 Tutorial 12.1: Noise Reduction Using Spatial-domain Techniques / 289 12.7 Problems / 296 13 MORPHOLOGICAL IMAGE PROCESSING 299 13.1 Introduction / 299 13.2 Fundamental Concepts and Operations / 300 13.2.1 The Structuring Element / 301 13.3 Dilation and Erosion / 304 13.3.1 Dilation / 305 13.3.2 Erosion / 307 13.4 Compound Operations / 310 13.4.1 Opening / 310 13.4.2 Closing / 311 13.4.3 Hit-or-Miss Transform / 313 13.5 Morphological Filtering / 314 13.6 Basic Morphological Algorithms / 315 13.6.1 Boundary Extraction / 317 13.6.2 Region Filling / 319 13.6.3 Extraction and Labeling of Connected Components / 321 13.7 Grayscale Morphology / 322 13.7.1 Dilation and Erosion / 323 13.7.2 Opening and Closing / 323 13.7.3 Top-Hat and Bottom-Hat Transformations / 325 13.8 Tutorial 13.1: Binary Morphological Image Processing / 325 13.9 Tutorial 13.2: Basic Morphological Algorithms / 330 13.10 Problems / 334 14 EDGE DETECTION 335 14.1 Formulation of the Problem / 335 14.2 Basic Concepts / 336 14.3 First-order Derivative Edge Detection / 338 14.4 Second-order Derivative Edge Detection / 343 14.4.1 Laplacian of Gaussian / 345 14.5 The Canny Edge Detector / 347 14.6 Edge Linking and Boundary Detection / 348 14.6.1 The Hough Transform / 349 14.7 Tutorial 14.1: Edge Detection / 354 14.8 Problems / 363 15 IMAGE SEGMENTATION 365 15.1 Introduction / 365 15.2 Intensity-based Segmentation / 367 15.2.1 Image Thresholding / 368 15.2.2 Global Thresholding / 369 15.2.3 The Impact of Illumination and Noise on Thresholding / 370 15.2.4 Local Thresholding / 371 15.3 Region-based Segmentation / 373 15.3.1 Region Growing / 374 15.3.2 Region Splitting and Merging / 377 15.4 Watershed Segmentation / 377 15.4.1 The Distance Transform / 378 15.5 Tutorial 15.1: Image Thresholding / 379 15.6 Problems / 386 16 COLOR IMAGE PROCESSING 387 16.1 The Psychophysics of Color / 387 16.1.1 Basic Concepts / 388 16.1.2 The CIE XYZ Chromaticity Diagram / 390 16.1.3 Perceptually Uniform Color Spaces / 393 16.1.4 ICC Profiles / 395 16.2 Color Models / 396 16.2.1 The RGB Color Model / 396 16.2.2 The CMY and CMYK Color Models / 398 16.2.3 The HSV Color Model / 398 16.2.4 The YIQ (NTSC) Color Model / 401 16.2.5 The YCbCr Color Model / 401 16.3 Representation of Color Images in MATLAB / 401 16.3.1 RGB Images / 402 16.3.2 Indexed Images / 403 16.4 Pseudocolor Image Processing / 406 16.4.1 Intensity Slicing / 406 16.4.2 Gray Level to Color Transformations / 407 16.4.3 Pseudocoloring in the Frequency Domain / 408 16.5 Full-color Image Processing / 409 16.5.1 Color Transformations / 410 16.5.2 Histogram Processing / 412 16.5.3 Color Image Smoothing and Sharpening / 412 16.5.4 Color Noise Reduction / 414 16.5.5 Color-Based Image Segmentation / 414 16.5.6 Color Edge Detection / 417 16.6 Tutorial 16.1: Pseudocolor Image Processing / 419 16.7 Tutorial 16.2: Full-color Image Processing / 420 16.8 Problems / 425 17 IMAGE COMPRESSION AND CODING 427 17.1 Introduction / 427 17.2 Basic Concepts / 428 17.2.1 Redundancy / 428 17.2.2 Image Encoding and Decoding Model / 431 17.3 Lossless and Lossy Compression Techniques / 432 17.3.1 Lossless Compression Techniques / 432 17.3.2 Lossy Compression Techniques / 433 17.4 Image Compression Standards / 435 17.4.1 Binary Image Compression Standards / 435 17.4.2 Continuous Tone Still Image Compression Standards / 435 17.4.3 JPEG / 436 17.4.4 JPEG 2000 / 437 17.4.5 JPEG-LS / 437 17.5 Image Quality Measures / 438 17.5.1 Subjective Quality Measurement / 438 17.5.2 Objective Quality Measurement / 439 17.6 Tutorial 17.1: Image Compression / 440 18 FEATURE EXTRACTION AND REPRESENTATION 447 18.1 Introduction / 447 18.2 Feature Vectors and Vector Spaces / 448 18.2.1 Invariance and Robustness / 449 18.3 Binary Object Features / 450 18.3.1 Area / 450 18.3.2 Centroid / 450 18.3.3 Axis of Least Second Moment / 451 18.3.4 Projections / 451 18.3.5 Euler Number / 452 18.3.6 Perimeter / 453 18.3.7 Thinness Ratio / 453 18.3.8 Eccentricity / 454 18.3.9 Aspect Ratio / 454 18.3.10 Moments / 455 18.4 Boundary Descriptors / 456 18.4.1 Chain Code, Freeman Code, and Shape Number / 459 18.4.2 Signatures / 461 18.4.3 Fourier Descriptors / 462 18.5 Histogram-based (Statistical) Features / 464 18.6 Texture Features / 466 18.7 Tutorial 18.1: Feature Extraction and Representation / 470 18.8 Problems / 474 19 VISUAL PATTERN RECOGNITION 475 19.1 Introduction / 475 19.2 Fundamentals / 476 19.2.1 Design and Implementation of a Visual Pattern Classifier / 476 19.2.2 Patterns and Pattern Classes / 478 19.2.3 Data Preprocessing / 479 19.2.4 Training and Test Sets / 480 19.2.5 Confusion Matrix / 480 19.2.6 System Errors / 481 19.2.7 Hit Rates, False Alarm Rates, and ROC Curves / 481 19.2.8 Precision and Recall / 482 19.2.9 Distance and Similarity Measures / 485 19.3 Statistical Pattern Classification Techniques / 487 19.3.1 Minimum Distance Classifier / 488 19.3.2 k -Nearest Neighbors Classifier / 490 19.3.3 Bayesian Classifier / 490 19.4 Tutorial 19.1: Pattern Classification / 491 19.5 Problems / 497 PART II VIDEO PROCESSING 20 VIDEO FUNDAMENTALS 501 20.1 Basic Concepts and Terminology / 501 20.2 Monochrome Analog Video / 507 20.2.1 Analog Video Raster / 507 20.2.2 Blanking Intervals / 508 20.2.3 Synchronization Signals / 509 20.2.4 Spectral Content of Composite Monochrome Analog Video / 509 20.3 Color in Video / 510 20.4 Analog Video Standards / 512 20.4.1 NTSC / 513 20.4.2 PAL / 513 20.4.3 SECAM / 514 20.4.4 HDTV / 514 20.5 Digital Video Basics / 514 20.5.1 Advantages of Digital Video / 515 20.5.2 Parameters of a Digital Video Sequence / 516 20.5.3 The Audio Component / 517 20.6 Analog-to-Digital Conversion / 517 20.7 Color Representation and Chroma Subsampling / 520 20.8 Digital Video Formats and Standards / 521 20.8.1 The Rec. 601 Digital Video Format / 522 20.8.2 The Common Intermediate Format / 523 20.8.3 The Source Intermediate Format / 524 20.9 Video Compression Techniques and Standards / 524 20.9.1 Video Compression Standards, Codecs, and Containers / 525 20.10 Video Processing in MATLAB / 526 20.10.1 Reading Video Files / 527 20.10.2 Processing Video Files / 527 20.10.3 Playing Video Files / 527 20.10.4 Writing Video Files / 528 20.11 Tutorial 20.1: Basic Digital Video Manipulation in MATLAB / 528 20.12 Tutorial 20.2: Working with YUV Video Data / 534 20.13 Problems / 539 21 VIDEO SAMPLING RATE AND STANDARDS CONVERSION 541 21.1 Video Sampling / 541 21.2 Sampling Rate Conversion / 542 21.3 Standards Conversion / 543 21.3.1 Deinterlacing / 543 21.3.2 Conversion between PAL and NTSC Signals / 545 21.3.3 Color Space Conversion / 545 21.3.4 Aspect Ratio Conversion / 546 21.3.5 3:2 Pull-Down / 547 21.4 Tutorial 21.1: Line Down-Conversion / 548 21.5 Tutorial 21.2: Deinterlacing / 550 21.6 Tutorial 21.3: NTSC to PAL Conversion / 556 21.7 Tutorial 21.4: 3:2 Pull-Down / 557 21.8 Problems / 559 22 DIGITAL VIDEO PROCESSING TECHNIQUES AND APPLICATIONS 561 22.1 Fundamentals of Motion Estimation and Motion Compensation / 561 22.2 General Methodologies in Motion Estimation / 564 22.2.1 Motion Representation / 566 22.2.2 Motion Estimation Criteria / 567 22.2.3 Optimization Methods / 567 22.3 Motion Estimation Algorithms / 568 22.3.1 Exhaustive Search Block Matching Algorithm / 568 22.3.2 Fast Algorithms / 570 22.3.3 Hierarchical Block Matching Algorithm / 571 22.3.4 Phase Correlation Method / 573 22.4 Video Enhancement and Noise Reduction / 573 22.4.1 Noise Reduction in Video / 574 22.4.2 Interframe Filtering Techniques / 575 22.5 Case Study: Object Segmentation and Tracking in the Presence of Complex Background / 576 22.6 Tutorial 22.1: Block-based Motion Estimation / 579 22.7 Tutorial 22.2: Intraframe and Interframe Filtering Techniques / 585 22.8 Problems / 589 Appendix A: HUMAN VISUAL PERCEPTION 591 A.1 Introduction / 591 A.2 The Human Eye / 592 A.3 Characteristics of Human Vision / 596 A.3.1 Resolution, Viewing Distance, and Viewing Angle / 596 A.3.2 Detail and Sharpness Perception / 598 A.3.3 Optical Transfer Function and Modulation Transfer Function / 599 A.3.4 Brightness Perception / 600 A.3.5 Contrast Ratio and Contrast Sensitivity Function / 603 A.3.6 Perception of Motion / 605 A.3.7 Spatiotemporal Resolution and Frequency Response / 606 A.3.8 Masking / 608 A.4 Implications and Applications of Knowledge about the Human Visual System / 609 Appendix B: GUI DEVELOPMENT 611 B.1 Introduction / 611 B.2 GUI File Structure / 611 B.3 Passing System Control / 613 B.4 The UserData Object / 615 B.5 A Working GUI Demo / 616 B.6 Concluding Remarks / 618 REFERENCES 619 INDEX 627