Classification, Parameter Estimation and State Estimation (eBook)

Professor Bangjun Lei, Dr. Guangzhu Xu, and Dr. Ming Feng are with The Institute of Intelligent Vision and Image Information, China Three Gorges University, China. Professor Yaobin Zou is an associate professor at China Three Gorges University. Dr. Ferdinand van der Heijden, Ph.D., is on the faculty of theDepartment of Signals and Systems, University of Twente, Netherlands. Professor Dick de Ridder is Professor at the Bioinformatics lab at Wageningen University, Netherlands. Professor David M. J. Tax, is a researcher with the Pattern Recognition laboratory, Delft University of Technology.

Preface xi

About the Companion Website xv

Introduction 1

1.1 The Scope of the Book 2

1.2 Engineering 10

1.3 The Organization of the Book 12

1.4 Changes from First Edition 14

1.5 References 15

PRTools Introduction 17

2.1 Motivation 17

2.2 Essential Concepts 18

2.3 PRTools Organization Structure and Implementation 22

2.4 Some Details about PRTools 26

2.5 Selected Bibliography 42

Detection and Classification 43

3.1 Bayesian Classification 46

3.2 Rejection 62

3.3 Detection:The Two-Class Case 66

3.4 Selected Bibliography 74

Exercises 74

Parameter Estimation 77

4.1 Bayesian Estimation 79

4.2 Performance Estimators 94

4.3 Data Fitting 100

4.4 Overview of the Family of Estimators 110

4.5 Selected Bibliography 111

Exercises 112

State Estimation 115

5.1 A General Framework for Online Estimation 117

5.2 Infinite Discrete-Time State Variables 125

5.3 Finite Discrete-Time State Variables 147

5.4 Mixed States and the Particle Filter 163

5.5 Genetic State Estimation 170

5.6 State Estimation in Practice 183

5.7 Selected Bibliography 201

Exercises 204

Supervised Learning 207

6.1 Training Sets 208

6.2 Parametric Learning 210

6.3 Non-parametric Learning 217

6.4 Adaptive Boosting - Adaboost 245

6.5 Convolutional Neural Networks (CNNs) 249

6.6 Empirical Evaluation 252

6.7 Selected Bibliography 257

Exercises 257

Feature Extraction and Selection 259

7.1 Criteria for Selection and Extraction 261

7.2 Feature Selection 272

7.3 Linear Feature Extraction 288

7.4 References 300

Exercises 300

Unsupervised Learning 303

8.1 Feature Reduction 304

8.2 Clustering 320

8.3 References 345

Exercises 346

Worked Out Examples 349

9.1 Example on Image Classification with PRTools 349

9.2 Boston Housing Classification Problem 361

9.3 Time-of-Flight Estimation of an Acoustic Tone Burst 372

9.4 Online Level Estimation in a Hydraulic System 392

9.5 References 406

Appendix A: Topics Selected from Functional Analysis 407

Appendix B: Topics Selected from Linear Algebra and Matrix Theory 421

Appendix C: Probability Theory 437

Appendix D: Discrete-Time Dynamic Systems 453

Index 459