Human Recognition at a Distance in Video (eBook)

Preface 5
Contents 8
List of Figures 13
List of Tables 21
Introduction to Gait-Based Individual Recognition at a Distance 24
Introduction 25
Gait-Based Human Recognition 26
Face-Based Human Recognition 26
Key Ideas Described in the Book 27
Organization of the Book 29
Gait-Based Individual Recognition at a Distance 32
Gait Representations in Video 33
Human Motion Analysis and Representations 33
Human Activity and Individual Recognition by Gait 34
Human Recognition by Gait 35
Model-Based Approaches 35
Model-Free Approaches 35
Human Activity Recognition 37
Model-Based Approaches 37
Model-Free Approaches 37
Gait Energy Image (GEI) Representation 37
Motivation 38
Representation Construction 38
Relationship with MEI and MHI 38
Representation Justification 39
Framework for GEI-Based Recognition 41
Silhouette Extraction and Processing 41
Feature Extraction 42
Summary 44
Model-Free Gait-Based Human Recognition in Video 45
Statistical Feature Fusion for Human Recognition by Gait 45
Real and Synthetic Gait Templates 46
Human Recognition 48
Experimental Results 50
Data and Parameters 50
Performance Evaluation 52
Human Recognition Based on Environmental Context 53
Walking Surface Type Detection 54
Classifier Design 57
Probabilistic Classifier Combination 58
Experimental Results 59
View-Insensitive Human Recognition by Gait 60
View-Insensitive Gait Templates 60
Human Recognition 62
Experimental Results 63
Human Repetitive Activity Recognition in Thermal Imagery 65
Object Detection in Thermal Infrared Imagery 66
Human Repetitive Activity Representation and Recognition 67
Experimental Results 68
Human Recognition Under Different Carrying Conditions 70
Technical Approach 70
Gait Energy Image (GEI) 70
Feature Extraction 71
Co-evolutionary Genetic Programming 72
Majority Voting 73
Experimental Results 73
Data 73
Experiments 74
Classifier Performance Comparison 74
Summary 75
Discrimination Analysis for Model-Based Gait Recognition 77
Predicting Human Recognition Performance 77
Algorithm Dependent Prediction and Performance Bounds 78
Body Part Length Distribution 78
Algorithm Dependent Performance Prediction 80
Upper Bound on PCR 81
Experimental Results 82
Summary 83
Model-Based Human Recognition-2D and 3D Gait 85
2D Gait Recognition (3D Model, 2D Data) 85
3D Human Modeling 86
Human Kinematic Model 86
Human Model Parameter Selection 87
Camera Model and Coordinate Transformation 88
World Coordinate to Camera Coordinate 89
Camera Coordinate to Ideal Image Coordinate 89
Ideal Image Coordinate to Actual Image Coordinate 89
Actual Image Coordinate to Computer Image Coordinate 90
Human Recognition from Single Non-calibrated Camera 90
Silhouette Preprocessing 90
Matching Between 3D Model and 2D Silhouette 91
Human Model Parameter Estimation 91
Stationary Parameter Estimation 91
Kinematic Parameter Estimation 92
Recognition Based on Kinematic and Stationary Features 93
Kinematic and Stationary Feature Classifier 93
Classifier Combination Strategies 93
Performance Evaluation on Monocular Image Sequences 94
Performance of Stationary Feature Classifier 94
Performance of Kinematic Feature Classifier 95
Performance with Classifier Combination 96
Human Recognition from Multiple Calibrated Cameras 96
Human Model Parameter Selection 96
Matching Between 3D Human Model and Multiple 2D Silhouettes 97
Human Model Parameter Initialization and Estimation 97
Performance Evaluation on Data from Multiple Cameras 98
Gait Recognition in 3D 100
Individual Recognition by Gait in 3D 100
Related Work 101
Technical Approach 103
3D Human Body Data 103
3D Human Body Model 104
Model Fitting 105
Body Axes 105
Torso 106
Arms and Legs 107
Head and Neck 108
Gait Reconstruction 108
Feature Matching 108
Experimental Results 109
Gait Reconstruction 109
Training and Testing Data 110
Gait Recognition 112
Summary 114
Fusion of Color/Infrared Video for Human Detection 115
Related Work 117
Hierarchical Image Registration and Fusion Approach 119
Image Transformation Model 120
Preliminary Human Silhouette Extraction and Correspondence Initialization 121
Automatic Image Registration 122
Model Parameter Selection 122
Parameter Estimation Based on Hierarchical Genetic Algorithm 124
Sensor Fusion 127
Registration of EO/IR Sequences with Multiple Objects 128
Experimental Results 128
Image Registration Results 129
Sensor Fusion Results 132
Summary 133
Face Recognition at a Distance in Video 135
Super-Resolution of Facial Images in Video at a Distance 136
Closed-Loop Super-Resolution of Face Images in Video 137
Related Work 137
Technical Approach 138
Bilinear Basis Images Computation 139
Pose and Illumination Estimation 139
Super-Resolution Algorithm 139
Experimental Results 141
Synthetic Data 141
Real Video 142
Super-Resolution of Facial Images with Expression Changes in Video 143
Related Work 144
Technical Approach 145
Tracking of Facial Regions 146
Local Deformation 147
Free Form Deformation Formulation 147
Cost Function 148
Resolution Aware Local Deformation 148
Deform Local Motion on High Resolution Data 149
Super-Resolution Methodology Requires Sub-pixel Registration 150
Super-Resolution Algorithm 151
A Match Measure for Warping Errors 151
Experimental Results 151
Data and Parameters 151
Results of Resolution Aware FFD 152
Super-Resolution Results-Global Registration vs. Global + RAIFFD Local Deformation 152
Quantification of Performance 153
Proposed Approach with Two Different SR Algorithms 155
Constructing Enhanced Side Face Images from Video 156
Enhanced Side Face Image (ESFI) Construction 158
Technical Approach 158
Acquiring Moving Head of a Person in Video 158
Side Face Image Alignment 158
Elastic Registration Method 158
Match Statistic 160
Resolution Enhancement Algorithm 162
The Imaging Model 162
Algorithm for Resolution Enhancement 163
Side Face Normalization 163
Summary 167
Evaluating Quality of Super-Resolved Face Images 168
Image Quality Indices 168
Integrated Image Quality Index 169
Gray Scale Based Quality (Qg) 171
Structure Based Quality (Qe) 172
Similarity Between Input Images (Qi) 173
Integrated Quality Measure (Qint) 174
Experimental Results for Face Recognition in Video 174
Experiment 1: Influence of Pose Variation on the Super-Resolved Face Image 175
Experiment 2: Influence of Lighting Variation on the Super-Resolved Face Image 177
Experiment 3: Influence of Facial Expression Variation on the Super-Resolved Face Image 178
Experiment 4: Influence of the Number of Images Used for Constructing the Super-Resolved Face Image for Face Recognition 179
Discussion 182
Summary 183
Integrated Face and Gait for Human Recognition at a Distance in Video 184
Integrating Face Profile and Gait at a Distance 185
Introduction 185
Technical Approach 187
High-Resolution Image Construction for Face Profile 187
The Imaging Model 188
The Super Resolution Algorithm 189
Face Profile Representation and Matching 191
Face Profile Extraction 192
Curvature-Based Fiducial Extraction 193
Profile Matching Using Dynamic Time Warping 194
Gait Recognition 196
Integrating Face Profile and Gait for Recognition at a Distance 197
Experimental Results 197
Face Profile-Based Recognition 197
Static Face Database 197
Experimental Results 198
Integrating Face Profile With Gait 199
Video Data 199
Experimental Results 199
Summary 202
Match Score Level Fusion of Face and Gait at a Distance 203
Introduction 204
Related Work 205
Technical Approach 206
Enhanced Side Face Image Construction 207
Gait Energy Image Construction 208
Human Recognition Using ESFI and GEI 208
Feature Learning Using PCA and MDA Combined Method 208
Recognition by Integrating ESFI and GEI 209
Experimental Results and Performance Analysis 211
Experiments and Parameters 211
Experiment 1 212
Experiment 2 216
Experiment 3 218
Performance Analysis 219
Discussion on Experiments 219
Performance Characterization Statistic Q 222
Summary 224
Feature Level Fusion of Face and Gait at a Distance 226
Introduction 226
Technical Approach 229
Human Identification Using ESFI and GEI 231
Feature Learning Using PCA 231
Synthetic Feature Generation and Classification 232
The Related Fusion Schemes 233
Fusion at the Match Score Level 234
Fusion at the Feature Level 235
Experimental Results and Comparisons 235
Experiments and Parameters 235
Experiment 1 239
Experiment 2 241
Discussion on Experiments 247
Summary 249
Conclusions for Integrated Gait and Face for Human Recognition at a Distance in Video 250
Conclusions and Future Work 251
Summary 251
Gait-Based Human Recognition at a Distance 251
Video-Based Human Recognition at a Distance 252
Fusion of Face and Gait for Human Recognition at Distance 253
Future Research Directions 254
References 256
Index 266