Human-Machine Interaction for Automated Vehicles (eBook)

Front Cover 1
Human-Machine Interaction for Automated Vehicles 4
Copyright Page 5
Contents 6
1. Introduction 12
1.1 Automation level 12
1.1.1 Traffic safety 13
1.1.2 Motivation 14
1.2 Summary of the book 15
References 16
2. Driver behaviour recognition based on eye gaze 18
2.1 Introduction 18
2.2 Methodology 21
2.2.1 Framework architecture 21
2.2.2 Gaze estimation 22
2.2.2.1 System framework 22
2.2.2.2 Video acquisition 23
2.2.2.3 Feature extraction 24
2.2.2.4 Feature mapping 25
2.2.2.5 Heat map visualisation 28
2.2.3 Object recognition 29
2.2.4 Activity classifier 31
2.3 Results 32
2.3.1 Gaze estimation 32
2.3.1.1 Indoor experiment 32
2.3.1.2 In-vehicle experiment 37
2.3.2 Object recognition 37
2.3.3 Nondriving-related activity identification and analysis 40
2.3.4 Comparison with the state-of-the-art 45
2.4 Discussion 46
2.5 Conclusions 49
References 50
3. Driver behaviour recognition based on hand-gesture 54
3.1 Introduction 54
3.2 Methodology 56
3.2.1 System architecture 56
3.2.2 Region of interest selection 58
3.2.3 Optical flow estimation 59
3.2.4 2-Stream convolutional neural network 61
3.2.5 Experiment setup and performance validation 64
3.3 Results 65
3.3.1 Two streams 65
3.3.2 Classification performance 67
3.3.3 Conflicted cases analysis 69
3.4 Discussion 72
3.5 Conclusion 73
References 74
4. Driver behaviour recognition based on head movement 78
4.1 Introduction 78
4.2 Methodology 80
4.2.1 Experiments 80
4.2.2 Time-varying correlation estimation 82
4.2.3 Feature selection and classification 84
4.3 Results 85
4.3.1 Phase 1 – training data 85
4.3.2 Phase 2 – testing data 89
4.4 Conclusion 93
References 94
5. Driver behaviour recognition based on the fusion of head movement and hand movement 96
5.1 Introduction 96
5.2 Related works 98
5.2.1 Non-driving-related activities’ recognition 98
5.2.2 3D convolutional neural network 99
5.3 Methodology 100
5.3.1 3D residual block 101
5.3.2 Architecture of the 3D convolutional neural network model 102
5.3.3 Prediction process for the framework 103
5.3.4 Visual explanations of convolutional neural network model predictions 103
5.4 Dataset and training 105
5.4.1 Experiment design 105
5.4.2 Camera setup 106
5.4.3 Data pre-processing 107
5.4.4 Training setup 108
5.5 Results 109
5.6 Visualisation and discussion 112
5.7 Conclusion 115
References 116
6. Real-time driver behaviour recognition 120
6.1 Introduction 120
6.2 Methodology 123
6.2.1 Inverted Linear Bottlenecks 124
6.2.2 Channel weighting and temporal weighting 125
6.2.3 Model structure 126
6.2.4 Saliency map visualisation 127
6.2.5 Dataset and pre-processing 128
6.2.6 Hardware 128
6.3 Results 129
6.3.1 Training 129
6.3.2 Results 131
6.3.3 Saliency map visualisation 134
6.4 Conclusion 138
References 138
7. The implication of non-driving tasks on the take-over process 142
7.1 Introduction 142
7.2 Methodology 144
7.2.1 Take-over concept 144
7.2.2 Experiment setup 145
7.2.2.1 Vehicle modification 145
7.2.2.2 Participants 146
7.2.2.3 Non-driving-related activities 146
7.2.2.4 Track and take-over scenarios 146
7.2.3 Data acquisition 147
7.3 Results 149
7.3.1 Road-checking behaviour analysis 149
7.3.2 Take-over performance 150
7.4 Conclusion 153
References 155
8. Driver workload estimation 158
8.1 Introduction 158
8.2 The hybrid methods 160
8.2.1 Methodology 160
8.2.2 Identification of important variables using error reduction ratio causality 161
8.2.3 Model estimation using support vector machine 164
8.3 Dataset and pre-processing 166
8.4 Results and discussions 168
8.4.1 Identification of important variables 168
8.4.2 Driver workload estimation 170
8.5 Conclusion 173
Appendix 174
References 174
9. Neuromuscular dynamics characterisation for human–machine interface 178
9.1 Introduction 178
9.2 Dynamic model of the human–machine interacting system 180
9.3 System identification methodology 181
9.3.1 Non-linear least squares problem formulation 182
9.3.2 The Gauss–Newton algorithm 183
9.3.3 Convergence analysis of the Gauss–Newton algorithm 183
9.4 Experiment design and data collection 186
9.4.1 Steering tasks 186
9.4.2 Driver posture and hand positions 187
9.4.3 Data measurement and acquisition 188
9.5 Experiment results 189
9.6 Result analysis and discussion 192
9.6.1 Comparison of the passive and active steering tasks 192
9.6.2 Comparison of the results with different hand positions 194
9.6.3 Comparison of the results with different postures 195
9.6.4 Correlation analysis 195
9.7 Conclusions and future work 196
References 197
10. Driver steering intention prediction using neuromuscular dynamics 200
10.1 Introduction 200
10.2 High-level architecture of the system 203
10.3 Experiment design and data analysis 206
10.3.1 Experiment design 206
10.3.2 Electromyography data analysis 208
10.4 Hybrid-learning-based time-series model 210
10.4.1 Model construction 210
10.4.2 Model training and implementation 214
10.5 Experiment results 215
10.5.1 Evaluation metrics and baselines 215
10.5.2 Continuous steering torque prediction 217
10.5.3 Discrete steering intention prediction 218
10.5.4 Discussions and future works 222
10.6 Conclusion 224
References 224
11. Intelligent haptic interface design for human–machine interaction in automated vehicles 228
11.1 Introduction 228
11.2 Experimental results 231
11.2.1 Take-over time 232
11.2.2 Driver steering torque 234
11.2.3 Steering wheel angle 234
11.2.4 Yaw rate of the vehicle 236
11.3 Discussion 236
11.4 Experimental methods 241
11.4.1 Experimental design 241
11.4.2 Take-over request signalling 242
11.4.3 Assessment of driver states and control ability 243
11.4.4 Driver control authority and performance 244
11.4.5 Modelling of human–machine collaboration process 245
11.4.6 The two-phase predictive haptic steering torque controller 246
11.4.7 Participants 248
11.4.8 Statistical analysis 249
References 249
Index 252
Back Cover 261