All of Nonparametric Statistics (eBook)

Preface 7
Contents 9
Introduction 13
1.1 What Is Nonparametric Inference? 13
1.2 Notation and Background 14
1.3 Confidence Sets 17
1.4 Useful Inequalities 20
1.5 Bibliographic Remarks 22
1.6 Exercises 22
Estimating the cdf and Statistical Functionals 24
2.1 The cdf 24
2.2 Estimating Statistical Functionals 26
2.3 Influence Functions 29
2.4 Empirical Probability Distributions 32
2.5 Bibliographic Remarks 34
2.6 Appendix 34
2.7 Exercises 35
The Bootstrap and the Jackknife 37
3.1 The Jackknife 37
3.2 The Bootstrap 40
3.3 Parametric Bootstrap 41
3.4 Bootstrap Confidence Intervals 42
3.5 Some Theory 45
3.6 Bibliographic Remarks 47
3.7 Appendix 47
3.8 Exercises 49
Smoothing: General Concepts 52
4.1 The Bias–Variance Tradeoff 59
4.2 Kernels 64
4.3 Which Loss Function? 66
4.4 Confidence Sets 66
4.5 The Curse of Dimensionality 67
4.6 Bibliographic Remarks 68
4.7 Exercises 68
Nonparametric Regression 70
5.1 Review of Linear and Logistic Regression 72
5.2 Linear Smoothers 75
5.3 Choosing the Smoothing Parameter 77
5.4 Local Regression 80
5.5 Penalized Regression, Regularization and Splines 90
5.6 Variance Estimation 94
5.7 Confidence Bands 98
5.8 Average Coverage 103
5.9 Summary of Linear Smoothing 104
5.10 Local Likelihood and Exponential Families 105
5.11 Scale-Space Smoothing 108
5.12 Multiple Regression 109
5.13 Other Issues 120
5.14 Bibliographic Remarks 128
5.15 Appendix 128
5.16 Exercises 129
Density Estimation 133
6.1 Cross-Validation 134
6.2 Histograms 135
6.3 Kernel Density Estimation 139
6.4 Local Polynomials 145
6.5 Multivariate Problems 146
6.6 Converting Density Estimation Into Regression 147
6.7 Bibliographic Remarks 148
6.8 Appendix 148
6.9 Exercises 150
Normal Means and Minimax Theory 153
7.1 The Normal Means Model 153
7.2 Function Spaces 155
7.3 Connection to Regression and Density Estimation 157
7.4 Stein’s Unbiased Risk Estimator (sure) 158
7.5 Minimax Risk and Pinsker’s Theorem 161
7.6 Linear Shrinkage and the James–Stein Estimator 163
7.7 Adaptive Estimation Over Sobolev Spaces 166
7.8 Confidence Sets 167
7.9 Optimality of Confidence Sets 174
7.10 Random Radius Bands? 178
7.11 Penalization, Oracles and Sparsity 179
7.12 Bibliographic Remarks 180
7.13 Appendix 181
7.14 Exercises 188
Nonparametric Inference Using Orthogonal Functions 191
8.1 Introduction 191
8.2 Nonparametric Regression 191
8.3 Irregular Designs 198
8.4 Density Estimation 200
8.5 Comparison of Methods 201
8.6 Tensor Product Models 201
8.7 Bibliographic Remarks 202
8.8 Exercises 202
Wavelets and Other Adaptive Methods 204
9.1 Haar Wavelets 206
9.2 Constructing Wavelets 210
9.3 Wavelet Regression 213
9.4 Wavelet Thresholding 215
9.5 Besov Spaces 218
9.6 Confidence Sets 221
9.7 Boundary Corrections and Unequally Spaced Data 222
9.8 Overcomplete Dictionaries 222
9.9 Other Adaptive Methods 223
9.10 Do Adaptive Methods Work? 227
9.11 Bibliographic Remarks 228
9.12 Appendix 228
9.13 Exercises 230
Other Topics 233
10.1 Measurement Error 233
10.2 Inverse Problems 239
10.3 Nonparametric Bayes 241
10.4 Semiparametric Inference 241
10.5 Correlated Errors 242
10.6 Classification 242
10.7 Sieves 243
10.8 Shape-Restricted Inference 243
10.9 Testing 244
10.10 Computational Issues 246
10.11 Exercises 246
Bibliography 249
Index 266