The Art and Science of HDR Imaging

John J. McCann, Consultant, McCann Imaging, USA John McCann received a B.A. degree in Biology from Harvard University in 1964. He worked in, and later managed, the Vision Research Laboratory at Polaroid from 1961 to 1996. He has studied human color vision, digital image processing, large format instant photography and the reproduction of fine art. His 120 publications have studied Retinex theory, color from rod/Lcone interactions at low light levels, appearance with scattered light, and HDR imaging. He has been a Fellow of the Society of Imaging Science and Technology (IS&T) since 1983. He is a past President of IS&T and the Artists Foundation, Boston. In 1996 he received the SID Certificate of Commendation. He is the IS&T/OSA 2002 Edwin H. Land Medalist, and IS&T 2005 Honorary Member, and is a 2008 Fellow of the Optical Society of America. He is currently consulting and continuing his research on color vision. Alessandro Rizzi, Università degli Studi di Milano, Italy Professor Alessandro Rizzi holds a degree in Computer Science at University of Milano and received a PhD in Information Engineering at University of Brescia (Italy). He taught Information Systems and Computer Graphics at University of Brescia and at Politecnico di Milano. He is currently an  assistant professor teaching Multimedia and Human-Computer Interaction, and senior research fellow at the Department of Information Technologies at University of Milano. Since 1990 he has researched in the field of digital imaging and vision. His main research topic is the use of color information in digital images with particular attention to color perception mechanisms. He is the coordinator of the Italian Color Group Conference Chair of Color Conference at IS&T/SPIE Electronic Imaging, and a principle organizer of European Marie Curie Project CREATE.

About the Authors xix Preface xxi

Series Preface xxiii

Acknowledgements xxv

Section A HISTORY OF HDR IMAGING 1

1 HDR Imaging 3

1.1 Topics 3

1.2 Introduction 3

1.3 Replicas and Reproductions 4

1.4 A Choice of Metaphors for HDR Reproduction 5

1.5 Reproduction of Scene Dynamic Range 7

1.6 HDR Disciplines 8

1.7 Outline of the Text 10

1.8 Summary 11

1.9 References 12

2 HDR Tools and Defi nitions 13

2.1 Topics 13

2.2 Introduction 13

2.3 Pixels 14

2.4 Dynamic Ranges 14

2.5 Measuring Light 17

2.6 Measuring Color Spaces 18

2.7 Image Reproduction 21

2.8 Contrast 24

2.9 Digital Imaging 25

2.10 Summary 25

2.11 References 26

3 HDR in Natural Scenes 27

3.1 Topics 27

3.2 Appearance in HDR and Color Constancy 27

3.3 Summary 30

3.4 References 31

4 HDR in Painting 33

4.1 Topics 33

4.2 Introduction 33

4.3 Ancient Painting 33

4.4 Perspective 35

4.5 Chiaroscuro 37

4.6 Gerritt van Honthorst (Gherardo delle Notti) 37

4.7 Rembrandt van Vijn 40

4.8 John Constable 40

4.9 John Martin 40

4.10 Impressionism 41

4.11 Photorealism 43

4.12 Summary 43

4.13 References 44

5 HDR in Film Photography 45

5.1 Topics 45

5.2 Introduction 45

5.3 Multiple Exposures in the 1850s 46

5.4 HP Robinson 47

5.5 Hurter and Driffi eld-Scientifi c Calibration of AgX Film Sensitivity 48

5.6 Sheppard and Mees 50

5.7 19th Century – Professional Amateur Photography 50

5.8 20th Century – Corporate Photography 50

5.9 20th Century Control of Dynamic Range 51

5.10 Other Silver-Halide Stories 56

5.11 Summary 56

5.12 References 57

6 The Ansel Adams Zone System 59

6.1 Topics 59

6.2 Introduction 59

6.3 Compressing the HDR World into the LDR Print 59

6.4 Visualization 60

6.5 Scene Capture 61

6.6 “Performing the Score” 65

6.7 Moonrise, Hernandez 66

6.8 Apparent vs. Physical Contrast 66

6.9 Summary 66

6.10 References 68

7 Electronic HDR Image Processing: Analog and Digital 69

7.1 Topics 69

7.2 Introduction 69

7.3 Human Spatial Vision 69

7.4 Electronic HDR Image Processing 70

7.5 Summary 74

7.6 References 75

8 HDR and the World of Computer Graphics 77

8.1 Topics 77

8.2 Introduction 77

8.3 Early Years: the 60s 78

8.4 Early Digital Image Synthesis: the 70s 78

8.5 The Turning Point: the 80s 79

8.6 Computational Photorealism: from the 90s 80

8.7 Summary 80

8.8 References 81

9 Review of HDR History 83

9.1 Topics 83

9.2 Summary of Disciplines 83

9.3 Review 84

9.4 Summary 87

9.5 References 87

Section B MEASURED DYNAMIC RANGES 89

10 Actual Dynamic Ranges 91

10.1 Topics 91

10.2 Introduction 91

10.3 Dynamic Range of Light Sensors 92

10.4 Bits per Pixel 93

10.5 Dynamic Range of Display Devices 94

10.6 Interactions of Pixels in Images 95

10.7 Summary 96

10.8 References 96

11 Limits of HDR Scene Capture 99

11.1 Topics 99

11.2 Introduction 99

11.3 HDR Test Targets 99

11.4 Camera Veiling Glare Limits 101

11.5 Glare in Film Cameras 107

11.6 Review 111

11.7 Summary 111

11.8 References 112

12 Limits of HDR in Humans 113

12.1 Topics 113

12.2 Introduction 113

12.3 Visual Appearance of HDR Displays 113

12.4 von Honthorst’s Painting and the 4scaleBlack HDR Target 116

12.5 HDR Displays and Black and White Mondrian 116

12.6 HDR and Tone Scale Maps 117

12.7 HDR Displays and Contrast 117

12.8 Summary 117

12.9 References 118

13 Why Does HDR Improve Images? 119

13.1 Topics 119

13.2 Introduction 119

13.3 Why are HDR Images Better? 120

13.4 Are Multiple Exposures Necessary? 120

13.5 Summary 121

13.6 References 121

Section C SEPARATING GLARE AND CONTRAST 123

14 Two Counteracting Mechanisms: Glare and Contrast 125

14.1 Topics 125

14.2 Introduction 125

14.3 Two Spatial Mechanisms 126

14.4 Calculated Retinal Image 126

14.5 Measuring the Range of HDR Appearances 131

14.6 Calculating the Retinal Image 131

14.7 Visualizing the Retinal Image 131

14.8 HDR and Uniform Color Space 132

14.9 Summary 132

14.10 References 132

15 Measuring the Range of HDR Appearances 135

15.1 Topics 135

15.2 Introduction 135

15.3 Design of Appearance Scale Target 136

15.4 Magnitude Estimation Experiments 138

15.5 Scene Dependent Tone Scale 141

15.6 Glare and Contrast 142

15.7 Summary 143

15.8 References 143

16 Calculating the Retinal Image 145

16.1 Topics 145

16.2 Introduction 145

16.3 Converting Scene Luminance to Retinal Contrast 146

16.4 Calculating Retinal Radiance 146

16.5 Changes in the Retinal Image from Glare 149

16.6 Appearance and Retinal Image 149

16.7 Scene Content and Psychometric Functions 151

16.8 Summary 151

16.9 References 152

17 Visualizing HDR Images 153

17.1 Topics 153

17.2 Introduction 153

17.3 Calculated Retinal Image Contrast 154

17.4 Retinal Image Contrast 155

17.5 Summary 159

17.6 References 159

18 HDR and Uniform Color Spaces 161

18.1 Topics 161

18.2 Introduction 161

18.3 Uniform Color Spaces – Psychophysics 161

18.4 Color Vision – Physiology 164

18.5 Accurate Transformations from CMF to UCS 165

18.6 Summary 167

18.7 References 168

19 Glare: A Major Part of Vision Theory 169

19.1 Topics 169

19.2 Introduction 169

19.3 Glare: Distorts Lightness below Middle Gray, More or Less 169

19.4 Pixel-based HDR Image Processing 170

19.5 Summary 171

19.6 References 171

Section D SCENE CONTENT CONTROLS APPEARANCE 173

20 Scene Dependent Appearance of Quanta Catch 175

20.1 Topics 175

20.2 Introduction 175

20.3 Models of Vision – A Choice of Paradigms 175

20.4 Illumination, Constancy and Surround 176

20.5 Maximum’s Enclosure and Distance 176

20.6 Size of Maxima 177

20.7 Assimilation 177

20.8 Maxima and Contrast with Maxima 177

21 Illumination, Constancy and Surround 179

21.1 Topics 179

21.2 Introduction 179

21.3 Hipparchus of Nicea 180

21.4 Flat-2-D Transparent Displays 182

21.5 A Simple Two-Step Physical Description 183

21.6 Complex 3-D Scenes 185

21.7 Local Maxima 189

21.8 Review 190

21.9 Summary 190

21.10 References 191

22 Maximum’s Enclosure and Separation 193

22.1 Topics 193

22.2 Introduction 193

22.3 Experimental Design 194

22.4 Lightness Matches – Light Gray on Black 194

22.5 Lightness Matches – Dark Gray on Black 195

22.6 Dark Gray on Black: Varying White’s Position 197

22.7 Review 198

22.8 Summary 199

22.9 References 200

23 Maxima Size and Distribution 201

23.1 Topics 201

23.2 Introduction 201

23.3 Experimental Procedure 202

23.4 Controls 202

23.5 Dispersion of White (“Snow”) 202

23.6 Sides and Corners 203

23.7 Lines 204

23.8 Equivalent Backgrounds 205

23.9 Equivalent Backgrounds and Models of Vision 207

23.10 Summary 207

23.11 References 208

24 From Contrast to Assimilation 209

24.1 Topics 209

24.2 Introduction 209

24.3 Segmented Surrounds 210

24.4 Checkerboard Variants 215

24.5 Summary 216

24.6 References 216

25 Maxima and Contrast with Maxima 217

25.1 Topics 217

25.2 Merger of Aperture and Object Modes 217

25.3 Infl uence of the Maxima 218

25.4 Summary 219

Section E COLOR HDR 221

26 HDR, Constancy and Spatial Content 223

26.1 Topics 223

26.2 Introduction 223

26.3 Red and White Projections 224

26.4 Color Mondrians 225

26.5 Constancy’s On/Off Switch 225

26.6 Color of 3-D Mondrians – LDR/HDR Illumination 226

26.7 Color Constancy is HDR 226

26.8 References 226

27 Color Mondrians 227

27.1 Topics 227

27.2 Introduction 227

27.3 Color Mondrians 229

27.4 The Signature of Color Constancy 237

27.5 Search for Evidence of Adaptation – Averages 240

27.6 Transparency in Mondrians 243

27.7 Color Assimilation 243

27.8 Summary 244

27.9 References 245

28 Constancy’s On/Off Switch 247

28.1 Topics 247

28.2 Introduction 247

28.3 Maximov’s Shoe Boxes 247

28.4 New Maxima Restores Constancy 250

28.5 Independent L, M, S Spatial Processing 251

28.6 Model Predictions 253

28.7 Center-Surround Target – Results 253

28.8 Summary 255

28.9 References 256

29 HDR and 3-D Mondrians 257

29.1 Topics 257

29.2 Color Constancy and Appearance 257

29.3 Color Constancy Models 258

29.4 Measuring Changes in Appearance from Changes in Illumination 259

29.5 Magnitude Estimation Appearance Measurements 262

29.6 Watercolor Rendition Measurements of Appearance 263

29.7 Review of 3-D Mondrian Psychophysical Measurements 266

29.8 Color Constancy Models 268

29.9 Conclusions 270

29.10 References 271

30 Color Constancy is HDR 273

30.1 Topics 273

30.2 Introduction 273

30.3 Rod Receptors and HDR 274

30.4 Assembling Appearance: Color Constancy, Rod Vision and HDR 279

30.5 Summary 280

30.6 References 280

Section F HDR IMAGE PROCESSING 283

31 HDR Pixel and Spatial Algorithms 285

31.1 Topics 285

31.2 Introduction – HDR Image Processing Algorithms 285

31.3 One Pixel – Tone Scale Curves 286

31.4 Some of the Pixels – Local Processing 288

31.5 All of the Pixels 289

31.6 All Pixels and Scene Dependent – The Retinex Extended Family 289

31.7 Retinex Algorithms 290

31.8 ACE Algorithms 290

31.9 Analytical, Computational and Variational Algorithms 290

31.10 Techniques for Analyzing HDR Algorithms 290

31.11 The HDR Story 291

31.12 References 291

32 Retinex Algorithms 293

32.1 Topics 293

32.2 Introduction 293

32.3 How to Calculate Lightness Using Ratio-Products 297

32.4 A Variety of Processing Networks 301

32.5 Image Content 302

32.6 Real Images – 1975 307

32.7 The Extended Family of Retinex Models 319

32.8 Algorithm’s Goal 334

32.9 References 337

33 ACE Algorithms 341

33.1 Topics 341

33.2 Introduction 341

33.3 ACE Algorithm 341

33.4 Retinex and ACE 344

33.5 ACE Characteristics 345

33.6 RACE 349

33.7 Other Vision-based Models 350

33.8 Summary 350

33.9 References 351

34 Analytical, Computational and Variational Algorithms 353

34.1 Topics 353

34.2 Introduction 353

34.3 Math in the Framework of the Human Visual System 354

34.4 Analytical Retinex Formulas 354

34.5 Computational Retinex in Wavelets 354

34.6 Retinex and the Variational Techniques 355

34.7 Summary 356

34.8 References 357

35 Evaluation of HDR Algorithms 359

35.1 Topics 359

35.2 Introduction 359

35.3 Quantitative Approaches to Algorithm Evaluation 360

35.4 Lightness Test Targets 361

35.5 Ratio Metric 362

35.6 Quantitative Evaluation of 3-D Mondrians 367

35.7 Locality Test Targets 369

35.8 Summary 370

35.9 Lessons From Quantitative Studies of HDR in Cameras 371

35.10 References 371

36 The HDR Story 373

36.1 Topic 373

36.2 Straightforward Technology Stories 373

36.3 The HDR Story is Defi ned by Limits 373

36.4 HDR Works Well 374

36.5 References 375

Glossary 377

Author Index 385

Subject Index 387