Multimodal Computational Attention for Scene Understanding and Robotics (eBook)

Foreword 6
Contents 8
About the Author 10
Abbreviations 10
List of Figures 13
List of Tables 20
Abstract 22
1.1 Contributions 27
1.2 Outline 29
References 30
2 Background 31
2.1 Attention Models 31
2.1.1 Visual Attention 32
2.1.2 Auditory Attention 38
2.1.3 Multimodal Attention 43
2.2 Applications of Attention Models 44
2.2.1 Image Processing and Computer Vision 45
2.2.2 Audio Processing 46
2.2.3 Robotics 46
2.2.4 Computer Graphics 47
2.2.5 Design, Marketing, and Advertisement 48
References 49
3 Bottom-Up Audio-Visual Attention for Scene Exploration 56
3.1 Related Work and Contributions 58
3.1.1 Spectral Visual Saliency 58
3.1.2 Visual Saliency and Color Spaces 60
3.1.3 Visual Saliency and Faces 61
3.1.4 Auditory Saliency 62
3.1.5 Audio-Visual Saliency-Based Exploration 62
3.1.6 Scene Analysis 64
3.2 Visual Attention 65
3.2.1 Spectral Visual Saliency 67
3.2.2 Color Space Decorrelation 84
3.2.3 Modeling the Influence of Faces 95
3.3 Auditory Attention 102
3.3.1 Auditory Novelty Detection 102
3.3.2 Evaluation 106
3.4 Saliency-Based Audio-Visual Exploration 108
3.4.1 Gaussian Proto-Object Model 109
3.4.2 Auditory Proto-Objects 109
3.4.3 Visual Proto-Objects 110
3.4.4 Audio-Visual Fusion and Inhibition 113
3.4.5 Evaluation 115
3.5 Multiobjective Exploration Path 119
3.5.1 Exploration Path 120
3.5.2 Exploration Strategies 120
3.5.3 Evaluation 122
3.6 Summary and Future Directions 126
References 128
4.1 Related Work and Contributions 137
4.1.1 Joint Attention 137
4.1.2 Visual Attention 140
4.2 Debiased Salient Object Detection 144
4.2.1 The MSRA Dataset 145
4.2.2 MSRA's Photographer Bias 146
4.2.3 Salient Object Detection 149
4.2.4 Debiased Salient Object Detection and Pointing 153
4.3 Focusing Computational Attention in Human-Robot Interaction 154
4.3.1 Pointing Gestures 156
4.3.2 Language 165
4.4 Gaze Following in Web Images 179
4.4.1 Approach 180
4.4.2 The Gaze@Flickr Dataset 181
4.4.3 Evaluation 183
4.5 Summary and Future Directions 188
References 190
5 Conclusion 196
5.1 Summary 196
5.2 Future Work 198
Appendix A Applications 200
Appendix B Dataset Overview 206
Appendix C Color Space Decorrelation: Full Evaluation 209
Appendix D Center Bias Integration Methods 216