Machine Learning in Computer Vision (eBook)

Contents 7
Foreword 13
Preface 15
1 INTRODUCTION 18
1. Research Issues on Learning in Computer Vision 19
2. Overview of the Book 23
3. Contributions 29
2 THEORY: PROBABILISTIC CLASSIFIERS 32
1. Introduction 32
2. Preliminaries and Notations 35
2.1 Maximum Likelihood Classification 35
2.2 Information Theory 36
2.3 Inequalities 37
3. Bayes Optimal Error and Entropy 37
4. Analysis of Classification Error of Estimated (Mismatched) Distribution 44
4.1 Hypothesis Testing Framework 45
4.2 Classification Framework 47
5. Density of Distributions 48
5.1 Distributional Density 50
5.2 Relating to Classification Error 54
6. Complex Probabilistic Models and Small Sample Effects 57
7. Summary 58
3 THEORY: GENERALIZATION BOUNDS 62
1. Introduction 62
2. Preliminaries 64
3. A Margin Distribution Based Bound 66
3.1 Proving the Margin Distribution Bound 66
4. Analysis 74
4.1 Comparison with Existing Bounds 76
5. Summary 81
4 THEORY: SEMI-SUPERVISED LEARNING 82
1. Introduction 82
2. Properties of Classification 84
3. Existing Literature 85
4. Semi-supervised Learning Using Maximum Likelihood Estimation 87
5. Asymptotic Properties of Maximum Likelihood Estimation with Labeled and Unlabeled Data 90
5.1 Model Is Correct 93
5.2 Model Is Incorrect 94
5.3 Examples: Unlabeled Data Degrading Performance with Discrete and Continuous Variables 97
5.4 Generating Examples: Performance Degradation with Univariate Distributions 100
5.5 Distribution of Asymptotic Classi.cation Error Bias 103
5.6 Short Summary 105
6. Learning with Finite Data 107
6.1 Experiments with Artificial Data 108
6.2 Can Unlabeled Data Help with Incorrect Models? Bias vs. Variance Effects and the Labeled-unlabeled Graphs 109
6.3 Detecting When Unlabeled Data Do Not Change the Estimates 114
6.4 Using Unlabeled Data to Detect Incorrect Modeling Assumptions 116
7. Concluding Remarks 117
5 ALGORITHM: MAXIMUM LIKELIHOOD MINIMUM ENTROPY HMM 120
1. Previous Work 120
2. Mutual Information, Bayes Optimal Error, Entropy, and Conditional Probability 122
3. Maximum Mutual Information HMMs 124
3.1 Discrete Maximum Mutual Information HMMs 125
3.2 Continuous Maximum Mutual Information HMMs 127
3.3 Unsupervised Case 128
4. Discussion 128
4.1 Convexity 128
4.2 Convergence 129
4.3 Maximum A-posteriori View of Maximum Mutual Information HMMs 129
5. Experimental Results 132
5.1 Synthetic Discrete Supervised Data 132
5.2 Speaker Detection 132
5.3 Protein Data 134
5.4 Real-time Emotion Data 134
6. Summary 134
6 ALGORITHM: MARGIN DISTRIBUTION OPTIMIZATION 136
1. Introduction 136
2. A Margin Distribution Based Bound 137
3. Existing Learning Algorithms 138
4. The Margin Distribution Optimization (MDO) Algorithm 142
4.1 Comparison with SVM and Boosting 143
4.2 Computational Issues 143
5. Experimental Evaluation 144
6. Conclusions 145
7 ALGORITHM: LEARNING THE STRUCTURE OF BAYESIAN NETWORK CLASSIFIERS 146
1. Introduction 146
2. Bayesian Network Classifiers 147
2.1 Naive Bayes Classifiers 149
2.2 Tree-Augmented Naive Bayes Classifiers 150
3. Switching between Models: Naive Bayes and TAN Classifiers 155
4. Learning the Structure of Bayesian Network Classifiers: Existing Approaches 157
4.1 Independence-based Methods 157
4.2 Likelihood and Bayesian Score-based Methods 159
5. Classification Driven Stochastic Structure Search 160
5.1 Stochastic Structure Search Algorithm 160
5.2 Adding VC Bound Factor to the Empirical Error Measure 162
6. Experiments 163
6.1 Results with Labeled Data 163
6.2 Results with Labeled and Unlabeled Data 164
7. Should Unlabeled Data Be Weighed Differently? 167
8. Active Learning 168
9. Concluding Remarks 170
8 APPLICATION: OFFICE ACTIVITY RECOGNITION 174
1. Context-Sensitive Systems 174
2. Towards Tractable and Robust Context Sensing 176
3. Layered Hidden Markov Models (LHMMs) 177
3.1 Approaches 178
3.2 Decomposition per Temporal Granularity 179
4. Implementation of SEER 181
4.1 Feature Extraction and Selection in SEER 181
4.2 Architecture of SEER 182
4.3 Learning in SEER 183
4.4 Classification in SEER 183
5. Experiments 183
5.1 Discussion 186
6. Related Representations 187
7. Summary 189
9 APPLICATION: MULTIMODAL EVENT DETECTION 192
1. Fusion Models: A Review 193
2. A Hierarchical Fusion Model 194
2.1 Working of the Model 195
2.2 The Duration Dependent Input Output Markov Model 196
3. Experimental Setup, Features, and Results 199
4. Summary 200
10 APPLICATION: FACIAL EXPRESSION RECOGNITION 204
1. Introduction 204
2. Human Emotion Research 206
2.1 Affective Human-computer Interaction 206
2.2 Theories of Emotion 207
2.3 Facial Expression Recognition Studies 209
3. Facial Expression Recognition System 214
3.1 Face Tracking and Feature Extraction 214
3.2 Bayesian Network Classifiers: Learning the “Structure” of the Facial Features 217
4. Experimental Analysis 218
4.1 Experimental Results with Labeled Data 221
4.1.1 Person-dependent Tests 222
4.1.2 Person-independent Tests 223
4.2 Experiments with Labeled and Unlabeled Data 224
5. Discussion 225
11 APPLICATION: BAYESIAN NETWORK CLASSIFIERS FOR FACE DETECTION 228
1. Introduction 228
2. Related Work 230
3. Applying Bayesian Network Classifiers to Face Detection 234
4. Experiments 235
5. Discussion 239
References 242
Index 254

Chapter 11

APPLICATION: BAYESIAN NETWORK CLASSIFIERS FOR FACE DETECTION (p.211)

Images containing faces are essential to intelligent vision-based human computer interaction. To buld fully automated systems that analyze the information contained in face images, robust and ef.cient face detection algorithms are required. Among the face detection methods, the ones based on learning algorithms have attracted much attention recently and have demonstrated excellent results.

This chapter presents a discussion on semi-supervised learning of probabilistic mixture model classi.ers for face detection. Based on our complete theoretical analysis of semi-supervised learning using maximum likelihood presented in Chapter 4 we discuss the possibility of structure learning of Bayesian networks for face detection. We show that learning the structure of Bayesian networks classi.ers enables learning of good classi.ers for face detection with a small labeled set and a large unlabeled set.

1. Introduction
Many of the recent applications designed for human-computer intelligent interaction applications have used the human face as an input. Systems that perform face tracking for various applications, facial expression recognition and pose estimation of faces all rely on detection of human faces in the video frames [Pentland, 2000]. The rapidly expanding research in face processing is based on the premise that information about user’s identity, state, and intend can be extracted from images and that computers can react accordingly, e.g., by observing a person’s facial expression.

In the last years, face and facial expression recognition have attracted much attention despite the fact that they have been studied for more than 20 years by psychophysicists, neuroscientists, and engineers. Many research demonstrations and commercial applications have been developed from these efforts. Given an arbitrary image, the goal of face detection is to automatically locate a human face in an image or video, if it is present. Face detection in a general setting is a challenging problem due to the variability in scale, location, orientation (up-right, rotated), and pose (frontal, pro.le). Facial expression, occlusion, and lighting conditions also change the overall apprearance of faces. Yang et al. [Yang et al., 2002] summarize in their comprehensive survey the challenges associated with face detection:

- Pose. The images of a face vary due to the relative camera-face pose (frontal, 45 degree, pro.le, upside down), and some facial features (e.g., an eye or the nose) may become partially or wholly occluded.

- Presence or absence of structural components. Facial features such as beards, mustaches, and glasses may or may not be present and there is a great deal of variability among these components including shape, color, and size.

- Facial expression. The appearance of faces is directly affected by the facial expression of the persons.

-Occlusion. Faces may be partially occluded by other objects. In an image with a group of people, some faces may partially occlude other faces.

- Image orientation. Face images directly vary for different rotations about the camera’s optical axis.

-Imaging conditions. When the image is formed, factors such as lighting (spectra, source distribution and intensity) and camera characteristics (sensor response, lenses) affect the appearance of a face.