Intelligent Video Surveillance Systems

Jean-Yves Dufour is Head of the "Vision Lab", Thales/CEA common research laboratory, Palaiseau, France.

Introduction xiii
Jean-Yves DUFOUR and Phlippe MOUTTOU
 
Chapter 1. Image Processing: Overview and Perspectives 1
Henri MAÎTRE

1.1. Half a century ago 1

1.2. The use of images 3

1.3. Strengths and weaknesses of image processing 4

1.3.1. What are these theoretical problems that image processing has been unable to overcome?  5

1.3.2. What are the problems that image processing has overcome? 5

1.4. What is left for the future? 6

1.5. Bibliography 9

Chapter 2. Focus on Railway Transport 13
Sébastien AMBELLOUIS and Jean-Luc BRUYELLE

2.1. Introduction. 13

2.2. Surveillance of railway infrastructures 15

2.2.1. Needs analysis 15

2.2.2. Which architectures? 16

2.2.3. Detection and analysis of complex events  17

2.2.4. Surveillance of outside infrastructures 20

2.3. Onboard surveillance 21

2.3.1. Surveillance of buses 22

2.3.2. Applications to railway transport 23

2.4. Conclusion 28

2.5. Bibliography 30

Chapter 3. A Posteriori Analysis for Investigative Purposes 33
Denis MARRAUD, Benjamin CÉPAS, Jean-François SULZER, Christianne MULAT and Florence SÈDES

3.1. Introduction 33

3.2. Requirements in tools for assisted investigation 34

3.2.1. Prevention and security 34

3.2.2. Information gathering 35

3.2.3. Inquiry 36

3.3. Collection and storage of data 36

3.3.1. Requirements in terms of standardization  37

3.3.2. Attempts at standardization (AFNOR and ISO) 37

3.4. Exploitation of the data 39

3.4.1. Content-based indexing 39

3.4.2. Assisted investigation tools 43

3.5. Conclusion 44

3.6. Bibliography 45

Chapter 4. Video Surveillance Cameras 47
Cédric LE BARZ and Thierry LAMARQUE

4.1. Introduction 47

4.2. Constraints 48

4.2.1. Financial constraints 48

4.2.2. Environmental constraints 49

4.3. Nature of the information captured 49

4.3.1. Spectral bands 50

4.3.2. 3D or “2D + Z” imaging 51

4.4. Video formats 53

4.5. Technologies 55

4.6. Interfaces: from analog to IP 57

4.6.1. From analog to digital 57

4.6.2. The advent of IP   59

4.6.3. Standards. 60

4.7. Smart cameras 61

4.8. Conclusion 62

4.9. Bibliography 63
 
Chapter 5. Video Compression Formats 65
Marc LENY and Didier NICHOLSON

5.1. Introduction 65

5.2. Video formats 66

5.2.1. Analog video signals 66

5.2.2. Digital video: standard definition 67

5.2.3. High definition  68

5.2.4. The CIF group of formats 69

5.3. Principles of video compression 70

5.3.1. Spatial redundancy 70

5.3.2. Temporal redundancy 73

5.4. Compression standards 74

5.4.1. MPEG-2 74

5.4.2. MPEG-4 Part 2 75

5.4.3. MPEG-4 Part 10/H.264 AVC 77

5.4.4. MPEG-4 Part 10/H.264 SVC 79

5.4.5. Motion JPEG 2000 80

5.4.6. Summary of the formats used in video surveillance  82

5.5. Conclusion 83

5.6. Bibliography 84

Chapter 6. Compressed Domain Analysis for Fast Activity Detection 87
Marc LENY

6.1. Introduction 87

6.2. Processing methods 88

6.2.1. Use of transformed coefficients in the frequency domain 88

6.2.2. Use of motion estimation 90

6.2.3. Hybrid approaches 91

6.3. Uses of analysis of the compressed domain 93

6.3.1. General architecture 94

6.3.2. Functions for which compressed domain analysis is reliable  96

6.3.3. Limitations. 97

6.4. Conclusion 100

6.5. Acronyms  101

6.6. Bibliography 101

Chapter 7. Detection of Objects of Interest  103
Yoann DHOME, Bertrand LUVISON, Thierry CHESNAIS, Rachid BELAROUSSI, Laurent LUCAT, Mohamed CHAOUCH and Patrick SAYD

7.1. Introduction. 103

7.2. Moving object detection 104

7.2.1. Object detection using background modeling 104

7.2.2. Motion-based detection of objects of interest 107

7.3. Detection by modeling of the objects of interest 109

7.3.1. Detection by geometric modeling  109

7.3.2. Detection by visual modeling. 111

7.4. Conclusion 117

7.5. Bibliography 118

Chapter 8. Tracking of Objects of Interest in a Sequence of Images  123
Simona MAGGIO, Jean-Emmanuel HAUGEARD, Boris MEDEN, Bertrand LUVISON, Romaric AUDIGIER, Brice BURGER and Quoc Cuong PHAM

8.1. Introduction 123

8.2. Representation of objects of interest and their associated

visual features 124

8.2.1. Geometry 124

8.2.2. Characteristics of appearance 125

8.3. Geometric workspaces 127

8.4. Object-tracking algorithms 127

8.4.1. Deterministic approaches 127

8.4.2. Probabilistic approaches 128

8.5. Updating of the appearance models 132

8.6. Multi-target tracking 135

8.6.1. MHT and JPDAF 135

8.6.2. MCMC and RJMCMC sampling techniques 136

8.6.3. Interactive filters, track graph 138

8.7. Object tracking using a PTZ camera 138

8.7.1. Object tracking using a single PTZ camera only 139

8.7.2. Object tracking using a PTZ camera coupled with a static camera 139

8.8. Conclusion 141

8.9. Bibliography 142

Chapter 9. Tracking Objects of Interest Through a Camera Network 147
Catherine ACHARD, Sébastien AMBELLOUIS, Boris MEDEN,Sébastien LEFEBVRE and Dung Nghi TRUONG CONG

9.1. Introduction 147

9.2. Tracking in a network of cameras whose fields of view overlap 148

9.2.1. Introduction and applications 148

9.2.2. Calibration and synchronization of a camera network 150

9.2.3. Description of the scene by multi-camera aggregation 153

9.3. Tracking through a network of cameras with non-overlapping

fields of view  155

9.3.1. Issues and applications 155

9.3.2. Geometric and/or photometric calibration of a camera

network 156

9.3.3. Reidentification of objects of interest in a camera network 157

9.3.4. Activity recognition/event detection in a camera network 160

9.4. Conclusion 161

9.5. Bibliography 161

Chapter 10. Biometric Techniques Applied to Video Surveillance 165
Bernadette DORIZZI and Samuel VINSON

10.1. Introduction 165

10.2. The databases used for evaluation166

10.2.1. NIST-Multiple Biometrics Grand Challenge

(NIST-MBGC) 167

10.2.2. Databases of faces 167

10.3. Facial recognition 168

10.3.1. Face detection 168

10.3.2. Face recognition in biometrics 169

10.3.3. Application to video surveillance 170

10.4. Iris recognition 173

10.4.1. Methods developed for biometrics 173

10.4.2. Application to video surveillance 174

10.4.3. Systems for iris capture in videos 176

10.4.4. Summary and perspectives 177

10.5. Research projects 177

10.6. Conclusion 178

10.7. Bibliography 179

Chapter 11. Vehicle Recognition in Video Surveillance 183
Stéphane HERBIN

11.1. Introduction 183

11.2. Specificity of the context 184

11.2.1. Particular objects 184

11.2.2. Complex integrated chains 185

11.3. Vehicle modeling 185

11.3.1. Wire models 186

11.3.2. Global textured models 187

11.3.3. Structured models 188

11.4. Exploitation of object models 189

11.4.1. A conventional sequential chain with limited performance  189

11.4.2. Improving shape extraction 190

11.4.3. Inferring 3D information. 191

11.4.4. Recognition without form extraction 192

11.4.5. Toward a finer description of vehicles 193

11.5. Increasing observability 194

11.5.1. Moving observer 194

11.5.2. Multiple observers 195

11.6. Performances 196

11.7. Conclusion 196

11.8. Bibliography 197

Chapter 12. Activity Recognition 201
Bernard BOULAY and François BRÉMOND

12.1. Introduction 201

12.2. State of the art 202

12.2.1. Levels of abstraction 202

12.2.2. Modeling and recognition of activities 203

12.2.3. Overview of the state of the art 206

12.3. Ontology 206

12.3.1. Objects of interest 207

12.3.2. Scenario models 208

12.3.3. Operators 209

12.3.4. Summary 210

12.4. Suggested approach: the ScReK system 210

12.5. Illustrations 212

12.5.1. Application at an airport 213

12.5.2. Modeling the behavior of elderly people  213

12.6. Conclusion 215

12.7. Bibliography 215

Chapter 13. Unsupervised Methods for Activity Analysis and Detection of Abnormal Events 219
Rémi EMONET and Jean-Marc ODOBEZ

13.1. Introduction 219

13.2. An example of a topic model: PLSA 221

13.2.1. Introduction 221

13.2.2. The PLSA model 221

13.2.3. PLSA applied to videos 223

13.3. PLSM and temporal models 226

13.3.1. PLSM model 226

13.3.2. Motifs extracted by PLSM 228

13.4. Applications: counting, anomaly detection 230

13.4.1. Counting 230

13.4.2. Anomaly detection 230

13.4.3. Sensor selection 231

13.4.4. Prediction and statistics 233

13.5. Conclusion 233

13.6. Bibliography 233

Chapter 14. Data Mining in a Video Database 235
Luis PATINO, Hamid BENHADDA and François BRÉMOND

14.1. Introduction 235

14.2. State of the art 236

Table of Contents xi

14.3. Pre-processing of the data 237

14.4. Activity analysis and automatic classification 238

14.4.1. Unsupervised learning of zones of activity 239

14.4.2. Definition of behaviors 242

14.4.3. Relational analysis 243

14.5. Results and evaluations 245

14.6. Conclusion 248

14.7. Bibliography 249

Chapter 15. Analysis of Crowded Scenes in Video 251
Mikel RODRIGUEZ, Josef SIVIC and Ivan LAPTEV

15.1. Introduction 251

15.2. Literature review 253

15.2.1. Crowd motion modeling and segmentation 253

15.2.2. Estimating density of people in a crowded scene 254

15.2.3. Crowd event modeling and recognition 255

15.2.4. Detecting and tracking in a crowded scene 256

15.3. Data-driven crowd analysis in videos 257

15.3.1. Off-line analysis of crowd video database 258

15.3.2. Matching 258

15.3.3. Transferring learned crowd behaviors 260

15.3.4. Experiments and results 260

15.4. Density-aware person detection and tracking in crowds 262

15.4.1. Crowd model 263

15.4.2. Tracking detections 264

15.4.3. Evaluation 265

15.5. Conclusions and directions for future research 268

15.6. Acknowledgments 268

15.7. Bibliography 269

Chapter 16. Detection of Visual Context 273
Hervé LE BORGNE and Aymen SHABOU

16.1. Introduction 273

16.2. State of the art of visual context detection 275

16.2.1. Overview 275

16.2.2. Visual description 276

16.2.3. Multiclass learning 278

16.3. Fast shared boosting 279

16.4. Experiments. 281

16.4.1. Detection of boats in the Panama Canal 281

16.4.2. Detection of the visual context in video surveillance   283

16.5. Conclusion 285

16.6. Bibliography 286

Chapter 17. Example of an Operational Evaluation Platform: PPSL  289
Stéphane BRAUDEL

17.1. Introduction 289

17.2. Use of video surveillance: approach and findings 290

17.3. Current use contexts and new operational concepts 292

17.4. Requirements in smart video processing 293

17.5. Conclusion 294

Chapter 18. Qualification and Evaluation of Performances 297
Bernard BOULAY, Jean-François GOUDOU and François BRÉMOND

18.1. Introduction 297

18.2. State of the art 298

18.2.1. Applications 298

18.2.2. Process 299

18.3. An evaluation program: ETISEO 303

18.3.1. Methodology 303

18.3.2. Metrics 305

18.3.3. Summary 307

18.4. Toward a more generic evaluation 309

18.4.1. Contrast 310

18.4.2. Shadows 312

18.5. The Quasper project 312

18.6. Conclusion 313

18.7. Bibliography 314

List of Authors 315

Index   321