Multi-Modal User Interactions in Controlled Environments (eBook)

Foreword 7
Preface 8
Acknowledgements 10
Contents 11
Chapter 1 ntroduction 14
1.1 Introduction 14
1.2 Objective 15
1.3 Practical Applications 16
1.4 Research Challenges 18
1.4.1 Event Detection 18
1.4.2 Flow Estimation 18
1.4.3 Gaze Estimation 19
1.4.4 Role of the Context 19
1.4.5 Societal Issues 20
1.5 Technical Contribution 20
1.6 How the Book is Organized 22
Chapter 2 Abnormal Event Detection 24
2.1 Introduction 24
2.2 Related Work 26
2.3 Proposed Approach 28
2.3.1 Low-Level Features 29
2.3.1.1 Motion Heat Map 29
2.3.1.2 Points of Interest 30
2.3.1.3 Tracking Points of Interest 32
2.3.1.4 33
2.3.2 Intermediate-Level Features 34
2.3.2.1 Motion Area Ratio 34
2.3.2.2 Direction Variance 35
2.3.2.3 Motion Magnitude Variance 35
2.3.2.4 Direction Histogram 35
2.3.2.5 Direction Map 36
2.3.2.6 Difference of Direction Map 37
2.3.3 Other Intermediate-Level Features 39
2.3.3.1 Motion Continuity Factor 39
2.3.3.2 Motion Description Factor 42
2.3.3.3 Motion Trajectory of the Blob 44
2.3.4 High-Level Features 46
2.3.4.1 Detecting Collapsing Events 46
2.3.4.2 Detecting Opposing Flow Event 49
2.4 Group Detection and Tracking 51
2.4.1 Detection and Tracking of PoIs 52
2.4.2 Direction and Magnitude Models 53
2.4.3 Block Clustering 54
2.4.4 Group Tracking 55
2.5 Detecting Multiple Flows and Events in a Crowd Scene 56
2.5.1 Multiple Flow Detection 56
2.5.2 Event Recognition 56
2.5.3 Running and Walking Events 57
2.5.4 Crowd Convergence and Divergence Events 57
2.5.5 Results 59
2.6 Method Adaptation to Context 61
2.6.1 Overview 61
2.6.2 Context Factors 61
2.6.3 Method Extensions 63
2.6.4 Experiments 64
2.6.4.1 Some examples 64
2.6.4.2 Data Set 64
2.6.4.3 Methodology 66
2.6.5 Results 68
2.7 Conclusion 69
Chapter 3 Flow Estimation 72
3.1 Introduction 72
3.2 Related Works 73
3.2.1 Methods based on Motion Detection and Analysis 74
3.2.1.1 Approach Proposed by Xu and al. [149] 75
3.2.1.2 Approach Proposed by Zhang and Chen [153] 76
3.2.2 Methods Based on Contour Analysis 80
3.2.2.1 Approach Proposed by Bozzoli and al. [19] 80
3.2.3 Template-Based Methods 85
3.2.3.1 Approach Proposed by Sidla and al. [120] 86
3.2.4 Stereovision-Based methods 88
3.2.4.1 Approach Proposed by Terada and al. [130] 89
3.2.5 Spatio-Temporal Methods 92
3.2.5.1 Approach Proposed by Albiol and al. [1] 94
3.2.6 Commercial Applications 96
3.2.6.1 Cognimatics 96
3.2.6.2 Infodev 97
3.2.6.3 Eurecam Sarl 98
3.2.7 Contribution 98
3.3 Approach Steps 99
3.3.1 Blob Detection 99
3.3.2 Count Estimation 104
3.4 Experiments and Results 106
3.5 Conclusion 110
Chapter 4 Estimation of Visual Gaze 112
4.1 Human Vision System 112
4.2 History of Gaze Tracking 114
4.3 Gaze Tracking Techniques 115
4.3.1 Intrusive Systems 115
4.3.1.1 Electro-Oculography 115
4.3.1.2 Contact Lenses with Magnetic Coils 115
4.3.1.3 Localization of the Limb 116
4.3.1.4 Analysis of Eye Images 116
4.3.2 Non-Intrusive Systems 119
4.4 Applications 119
4.4.1 Interaction During Meetings 119
4.4.2 Driver Monitoring 120
4.4.3 Virtual Reality 120
4.4.4 Human Computer Interaction 121
4.4.5 Extraction of Saliency Maps in Images 122
4.4.6 Store Marketing 123
4.5 Contribution of Head Pose in Visual Gaze 123
4.5.1 Database 124
4.5.2 Calculating the Contribution of Head Pose 125
4.5.3 Prediction of the Target 125
4.6 Estimating Gaze Direction Based on Eye Localization Only 127
4.7 Head Pose Estimation 127
4.7.1 State of the Art 128
4.7.1.1 Definition 128
4.7.1.2 Human Capacity for Estimating Head Orientation 129
4.7.1.3 Problems Encountered when Estimating Head Poses 131
4.7.2 Image Datasets 140
4.7.2.1 Building Image Datasets 140
4.7.2.2 Utilized Image Database 142
4.7.3 Estimation of Head Pose Based on Global Appearance 144
4.7.3.1 Utilized Image Dataset 144
4.7.3.2 Feature Selection 145
4.7.3.3 Experimental Results 150
4.7.4 Cylindrical Model for Head Tracking 151
4.8 Conclusion 154
Chapter 5 Visual Field Projection and Region of Interest Analysis 156
5.1 Visual Field Estimation 156
5.1.1 Physiological Data 157
5.1.2 Visual Field Estimation and Fixation Point for Frontal Pose 158
5.1.3 Visual Field Adaptation to Head Orientation 160
5.1.3.1 Matrix Approach 161
5.2 Visual Field Projection 166
5.2.1 Point Projection 166
5.2.2 Perception Volume Projection 168
5.3 Visual Field Display and Projection on an Image 170
5.4 Region-of-Interest Extraction 172
5.4.1 Representation of Gaze Information 172
5.4.2 Gaze Point Correction 173
5.4.3 Calculation of Tilt and Pan Angles Corresponding to a Gaze Point 175
5.5 Metrics for Gaze Analysis 176
5.5.1 Construction of a System Measuring Media Relevance 176
5.5.1.1 Raw Data Collection 177
5.5.1.2 Identification of Fixations 177
5.5.2 Metrics Related to Fixation Distribution 178
5.5.3 Experiment 180
5.5.3.1 Images 180
5.5.3.2 Videos 181
5.5.4 Discussions 182
5.6 Conclusion 183
Chapter 6 Conclusion 184
6.1 Challenge 184
6.2 Perspectives 185
References 188
Appendix A Societal Recommendations 200
A.1 Societal Recommendations 201
A.1.1 Public Awareness 201
A.1.1.1 Observation and Surveillance Technology Awareness 201
A.1.1.2 Legal Awareness 202
A.1.1.3 Recommendation 1 202
A.1.2 Public Policy of Research and Development 202
A.1.2.1 EU R& D Awareness
A.1.2.2 Recommendation 2 203
A.1.2.3 Democratic Discussion 203
A.1.2.4 Recommendation 3 204
A.1.2.5 R& D Program Evaluation
A.1.2.6 Recommendation 4 204
A.1.2.7 Management and Human Science Researchers’ Role 205
A.1.2.8 Recommendation 5 205
A.1.3 Democratic Requirement for OST Regulation 205
A.1.3.1 Reinforcement of Public Authorities’ Assets with Regard to the Protection of Privacy and Data Protection 205
A.1.3.2 Recommendation 6 206
A.1.3.3 Intelligibility of the OST Systems 206
A.1.3.4 Accessibility to the OST Systems 206
A.1.3.5 Recommendation 7 207
A.1.3.6 Legitimacy of the OST’s Finalities 207
A.1.3.7 Recommendation 8 207
A.1.3.8 Recommendation 9 207
A.1.3.9 Privatization of Public Issues 208
A.1.3.10 Recommendation 10 208
A.2 Legal Recommendations 208
A.2.1 Data Protection and Privacy Issues 208
A.2.1.1 Recommendation 11 208
A.2.1.2 Recommendation 12 209
A.2.1.3 Recommendation 13 209
A.2.1.4 Recommendation 14 209
A.2.1.5 Recommendation 15 210
A.2.1.6 Recommendation 16 210
A.2.1.7 Recommendation 17 211
A.2.1.8 Recommendation 18 211
A.2.1.9 Recommendation 19 211
A.2.1.10 Recommendation 20 211
A.2.1.11 Recommendation 21 212
A.2.2 Beyond Data Protection 212
A.2.2.1 Data Protection and Fundamental Liberties 212
A.2.2.2 Recommendation 22 212
A.2.2.3 Recommendation 23 213
A.2.2.4 Recommendation 24 213
A.2.2.5 Specific Provisions about Terminals and Infrastructures 213
A.2.2.6 Recommendation 25 213
A.2.2.7 Recommendation 26 214
A.2.2.8 Recommendation 27 214
A.2.2.9 Recommendation 28 214
A.2.2.10 Recommendation 29 214
A.2.2.11 Recommendation 30 215
A.2.2.12 Towards a Regulation of Profiling Activities 215
A.2.2.13 Recommendation 31 215
A.2.2.14 Recommendation 32 215
A.2.2.15 Recommendation 33 216
A.2.2.16 Recommendation 34 216
A.2.2.17 Recommendation 35 216
A.2.3 Consumer Protection 216
A.2.3.1 Recommendation 36 216
A.2.3.2 Recommendation 37 217
A.2.3.3 Recommendation 38 217
A.2.3.4 Recommendation 39 217
Glossary 218
Index 220