Introduction to Statistical Pattern Recognition (eBook)

Cover 1
Frontmatter 5
Half Title Page 5
Title Page 7
Copyright 8
Dedication 9
Table of Contents 11
Preface 15
Acknowledgments 17
Chapter 1: Introduction 19
1.1 Formulation of Pattern Recognition Problems 19
1.2 Process of Classifier Design 25
Notation 27
References 28
Chapter 2: Random Vectors and Their Properties 29
2.1 Random Vectors and Their Distributions 29
2.2 Estimation of Parameters 35
2.3 Linear Transformation 42
2.4 Various Properties of Eigenvalues and Eigenvectors 53
Computer Projects 65
Problems 66
References 68
Chapter 3: Hypothesis Testing 69
3.1 Hypothesis Tests for Two Classes 69
3.2 Other Hypothesis Tests 83
3.3 Error Probability in Hypothesis Testing 103
3.4 Upper Bounds on the Bayes Error 115
3.5 Sequential Hypothesis Testing 128
Computer Projects 137
Problems 138
References 140
Chapter 4: Parametric Classifiers 142
4.1 The Bayes Linear Classifier 143
4.2 Linear Classifier Design 149
4.3 Quadratic Classifier Design 171
4.4 Other Classifiers 187
Computer Projects 194
Problems 195
References 198
Chapter 5: Parameter Estimation 199
5.1 Effect of Sample Size in Estimation 200
5.2 Estimation of Classification Errors 214
5.3 Holdout, Leave-One-Out, and Resubstitution Methods 237
5.4 Bootstrap Methods 256
Computer Projects 268
Problems 268
References 270
Chapter 6: Nonparametric Density Estimation 272
6.1 Parzen Density Estimate 273
6.2 k Nearest Neighbor Density Estimate 286
6.3 Expansion by Basis Functions 305
Computer Projects 313
Problems 314
References 315
Chapter 7: Nonparametric Classification and Error Estimation 318
7.1 General Discussion 319
7.2 Voting kNN Procedure - Asymptotic Analysis 323
7.3 Voting kNN Procedure - Finite Sample Analysis 331
7.4 Error Estimation 340
7.5 Miscellaneous Topics in the kNN Approach 369
Computer Projects 380
Problems 381
References 382
Chapter 8: Successive Parameter Estimation 385
8.1 Successive Adjustment of a Linear Classifier 385
8.2 Stochastic Approximation 393
8.3 Successive Bayes Estimation 407
Computer Projects 413
Problems 414
References 415
Chapter 9: Feature Extraction and Linear Mapping for Signal Representation 417
9.1 The Discrete Karhunen-Loéve Expansion 418
9.2 The Karhunen-Loéve Expansion for Random Processes 435
9.3 Estimation of Eigenvalues and Eigenvectors 443
Computer Projects 453
Problems 456
References 458
Chapter 10: Feature Extraction and Linear Mapping for Classification 459
10.1 General Problem Formulation 460
10.2 Discriminant Analysis 463
10.3 Generalized Criteria 478
10.4 Nonparametric Discriminant Analysis 484
10.5 Sequential Selection of Quadratic Features 498
10.5 Feature Subset Selection 507
Computer Projects 521
Problems 522
References 524
Chapter 11: Clustering 526
11.1 Parametric Clustering 527
11.2 Nonparametric Clustering 551
11.3 Selection of Representatives 567
Computer Projects 577
Problems 578
References 580
Backmatter 582
Appendix A: Derivatives of Matrices 582
Appendix B: Mathematical Formulas 590
Appendix C: Normal Error Table 594
Appendix D: Gamma Function Table 596
Index 597
About the Author 615
Back Cover 616