Information Criteria and Statistical Modeling (eBook)

Preface 6
Contents 9
1 Concept of Statistical Modeling 13
1.1 Role of Statistical Models 13
1.2 Constructing Statistical Models 16
1.3 Organization of This Book 19
2 Statistical Models 21
2.1 Modeling of Probabilistic Events and Statistical Models 21
2.2 Probability Distribution Models 22
2.3 Conditional Distribution Models 29
3 Information Criterion 41
3.1 Kullback–Leibler Information 41
3.2 Expected Log-Likelihood and Corresponding Estimator 47
3.3 Maximum Likelihood Method and Maximum Likelihood Estimators 49
3.4 Information Criterion AIC 63

4 Statistical Modeling by AIC 87
4.1 Checking the Equality of Two Discrete Distributions 87
4.2 Determining the Bin Size of a Histogram 89
4.3 Equality of the Means and/or the Variances of Normal Distributions 91
4.4 Variable Selection for Regression Model 96
4.5 Generalized Linear Models 100
4.6 Selection of Order of Autoregressive Model 104
4.7 Detection of Structural Changes 108
4.8 Comparison of Shapes of Distributions 113
4.9 Selection of Box–Cox Transformations 116
5 Generalized Information Criterion (GIC) 119
5.1 Approach Based on Statistical Functionals 119
5.2 Generalized Information Criterion (GIC) 130
6 Statistical Modeling by GIC 151
6.1 Nonlinear Regression Modeling via Basis Expansions 151
6.2 Basis Functions 155
6.3 Logistic Regression Models for Discrete Data 161
6.4 Logistic Discriminant Analysis 168
6.5 Penalized Least Squares Methods 172
6.6 Effective Number of Parameters 174
7 Theoretical Development and Asymptotic Properties of the GIC 178
7.1 Derivation of the GIC 178
7.2 Asymptotic Properties and Higher-Order Bias Correction 187
8 Bootstrap Information Criterion 197
8.1 Bootstrap Method 197
8.2 Bootstrap Information Criterion 202
8.3 Variance Reduction Method 205
8.4 Applications of Bootstrap Information Criterion 216
9 Bayesian Information Criteria 220
9.1 Bayesian Model Evaluation Criterion (BIC) 220
9.2 Akaike’s Bayesian Information Criterion (ABIC) 231
9.3 Bayesian Predictive Distributions 233
9.4 Bayesian Predictive Distributions by Laplace Approximation 240
9.5 Deviance Information Criterion (DIC) 245
10 Various Model Evaluation Criteria 247
10.1 Cross-Validation 247
10.2 Final Prediction Error (FPE) 255
10.3 Mallows’ Cp 259
10.4 Hannan–Quinn’s Criterion 261
10.5 ICOMP 262
References 263
Index 276