Modern Approach to Regression with R (eBook)

Preface 7
Contents 11
Introduction 15
1.1 Building Valid Models 15
1.2 Motivating Examples 15
1.2.1 Assessing the Ability of NFL Kickers 15
1.2.2 Newspaper Circulation 18
1.2.3 Menu Pricing in a New Italian Restaurant in New York City 19
1.2.4 Effect of Wine Critics’ Ratings on Prices of Bordeaux Wines 22
1.3 Level of Mathematics 27
Simple Linear Regression 29
2.1 Introduction and Least Squares Estimates 29
2.1.1 Simple Linear Regression Models 29
2.2 Inferences About the Slope and the Intercept 34
2.2.1 Assumptions Necessary in Order to Make Inferences About the Regression Model 35
2.2.2 Inferences About the Slope of the Regression Line 35
2.2.3 Inferences About the Intercept of the Regression Line 37
2.3 Confidence Intervals for the Population Regression Line 38
2.4 Prediction Intervals for the Actual Value of Y 39
2.5 Analysis of Variance 41
2.6 Dummy Variable Regression 44
2.7 Derivations of Results 47
2.7.1 Inferences about the Slope of the Regression Line 48
2.7.2 Inferences about the Intercept of the Regression Line 49
2.7.3 Confidence Intervals for the Population Regression Line 50
2.7.4 Prediction Intervals for the Actual Value of Y 51
2.8 Exercises 52
Diagnostics and Transformations for Simple Linear Regression 58
3.1 Valid and Invalid Regression Models: Anscombe’s Four Data Sets 58
3.1.1 Residuals 61
3.1.2 Using Plots of Residuals to Determine Whether the Proposed Regression Model Is a Valid Model 62
3.1.3 Example of a Quadratic Model 63
3.2 Regression Diagnostics: Tools for Checking the Validity of a Model 63
3.2.1 Leverage Points 64
3.2.2 Standardized Residuals 72
3.2.3 Recommendations for Handling Outliers and Leverage Points 79
3.2.4 Assessing the Influence of Certain Cases 80
3.2.5 Normality of the Errors 82
3.2.6 Constant Variance 84
3.3 Transformations 89
3.3.1 Using Transformations to Stabilize Variance 89
3.3.2 Using Logarithms to Estimate Percentage Effects 92
3.3.3 Using Transformations to Overcome Problems due to Nonlinearity 96
3.4 Exercises 116
Weighted Least Squares 127
4.1 Straight-Line Regression Based on Weighted Least Squares 127
4.1.1 Prediction Intervals for Weighted Least Squares 130
4.1.2 Leverage for Weighted Least Squares 130
4.1.3 Using Least Squares to Calculate Weighted Least Squares 131
4.1.4 Defining Residuals for Weighted Least Squares 133
4.1.5 The Use of Weighted Least Squares 133
4.2 Exercises 134
Multiple Linear Regression 136
5.1 Polynomial Regression 136
5.2 Estimation and Inference in Multiple Linear Regression 141
5.3 Analysis of Covariance 151
5.4 Exercises 157
Diagnostics and Transformations for Multiple Linear Regression 161
6.1 Regression Diagnostics for Multiple Regression 161
6.1.1 Leverage Points in Multiple Regression 162
6.1.2 Properties of Residuals in Multiple Regression 164
6.1.3 Added Variable Plots 172
6.2 Transformations 177
6.2.1 Using Transformations to Overcome Nonlinearity 177
6.2.2 Using Logarithms to Estimate Percentage Effects: Real Valued Predictor Variables 194
6.3 Graphical Assessment of the Mean Function Using Marginal Model Plots 199
6.4 Multicollinearity 205
6.4.1 Multicollinearity and Variance Inflation Factors 213
6.5 Case Study: Effect of Wine Critics’ Ratings on Prices of Bordeaux Wines 213
6.6 Pitfalls of Observational Studies Due to Omitted Variables 220
6.6.1 Spurious Correlation Due to Omitted Variables 220
6.6.2 The Mathematics of Omitted Variables 223
6.6.3 Omitted Variables in Observational Studies 224
6.7 Exercises 225
Variable Selection 236
7.1 Evaluating Potential Subsets of Predictor Variables 237
7.1.1 Criterion 1: R 237
-Adjusted 237
7.1.2 Criterion 2: AIC, Akaike’s Information Criterion 239
7.1.3 Criterion 3: AIC 240
, Corrected AIC 240
7.1.4 Criterion 4: BIC, Bayesian Information Criterion 241
7.1.5 Comparison of AIC, AIC 241
and BIC 241
7.2 Deciding on the Collection of Potential Subsets of Predictor Variables 242
7.2.1 All Possible Subsets 242
7.2.2 Stepwise Subsets 245
7.2.3 Inference After Variable Selection 247
7.3 Assessing the Predictive Ability of Regression Models 248
7.3.1 Stage 1: Model Building Using the Training Data Set 248
7.3.2 Stage 2: Model Comparison Using the Test Data Set 256
7.4 Recent Developments in Variable Selection – LASSO 259
7.5 Exercises 261
Logistic Regression 271
8.1 Logistic Regression Based on a Single Predictor 271
8.1.1 The Logistic Function and Odds 273
8.1.2 Likelihood for Logistic Regression with a Single Predictor 276
8.1.3 Explanation of Deviance 279
8.1.4 Using Differences in Deviance Values to Compare Models 280
8.1.5 R 281
for Logistic Regression 281
8.1.6 Residuals for Logistic Regression 282
8.2 Binary Logistic Regression 285
8.2.1 Deviance for the Case of Binary Data 288
8.2.2 Residuals for Binary Data 289
8.2.3 Transforming Predictors in Logistic Regression for Binary Data 290
8.2.4 Marginal Model Plots for Binary Data 294
8.3 Exercises 302
Serially Correlated Errors 312
9.1 Autocorrelation 312
9.2 Using Generalized Least Squares When the Errors Are AR( 1) 317
9.2.1 Generalized Least Squares Estimation 318
9.2.2 Transforming a Model with AR(1) Errors into a Model with iid Errors 322
9.2.3 A General Approach to Transforming GLS into LS 323
9.3 Case Study 326
9.4 Exercises 332
Mixed Models 337
10.1 Random Effects 337
10.1.1 Maximum Likelihood and Restricted Maximum Likelihood 340
10.1.2 Residuals in Mixed Models 351
10.2 Models with Covariance Structures Which Vary Over Time 359
10.2.1 Modeling the Conditional Mean 360
10.3 Exercises 374
Appendix: Nonparametric Smoothing 376
A.1 Kernel Density Estimation 376
A.2 Nonparametric Regression for a Single Predictor 379
A.2.1 Local Polynomial Kernel Methods 380
A.2.2 Penalized Linear Regression Splines 384
Index 392