Introduction to Time Series Analysis and Forecasting

Douglas C. Montgomery, PhD, is Regents Professor of Industrial Engineering and ASU Foundation Professor of Engineering at Arizona State University, USA. He holds a PhD in Engineering from Virginia Tech and has researched and published extensively on industrial statistics and experimental design. Cheryl Jennings, PhD, is Associate Teaching Professor at Arizona State University. She has decades of industrial experience in manufacturing and financial services, and has taught undergraduate and graduate courses on modeling and analysis, performance management, process control, and related subjects. Murat Kulahci, PhD, is Professor of Industrial Statistics at the Technical University of Denmark and Professor at the Luleå University of Technology, Sweden. He holds a PhD in Industrial Engineering from the University of Wisconsin, Madison. He has published widely on time series analysis, experimental design, process monitoring and related subjects.

Preface xi

About the Companion Website xv

1 Introduction to Time Series Analysis and Forecasting 1

1.1 The Nature and Uses of Forecasts 1

1.2 Some Examples of Time Series 9

1.3 The Forecasting Process 16

1.4 Data for Forecasting 19

1.4.1 The Data Warehouse 19

1.4.2 Data Wrangling and Cleaning 21

1.4.3 Imputation 22

1.5 Resources for Forecasting 23

Exercises 24

2 Statistics Background for Time Series Analysis and Forecasting 27

2.1 Introduction 27

2.2 Graphical Displays 28

2.2.1 Time Series Plots 28

2.2.2 Plotting Smoothed Data 32

2.3 Numerical Description of Time Series Data 37

2.3.1 Stationary Time Series 37

2.3.2 Autocovariance and Autocorrelation Functions 39

2.3.3 The Variogram 45

2.4 Use of Data Transformations and Adjustments 49

2.4.1 Transformations 49

2.4.2 Trend and Seasonal Adjustments 51

2.5 General Approach to Time Series Modeling and Forecasting 65

2.6 Evaluating and Monitoring Forecasting Model Performance 69

2.6.1 Forecasting Model Evaluation 69

2.6.2 Choosing Between Competing Models 78

2.6.3 Monitoring a Forecasting Model 81

2.7 R Commands for Chapter 2 89

Exercises 101

3 Regression Analysis and Forecasting 115

3.1 Introduction 115

3.2 Least Squares Estimation in Linear Regression Models 118

3.3 Statistical Inference in Linear Regression 127

3.3.1 Test for Significance of Regression 128

3.3.2 Tests on Individual Regression Coefficients and Groups of Coefficients 131

3.3.3 Confidence Intervals on Individual Regression Coefficients 137

3.3.4 Confidence Intervals on the Mean Response 138

3.4 Prediction of New Observations 141

3.5 Model Adequacy Checking 143

3.5.1 Residual Plots 143

3.5.2 Scaled Residuals and PRESS 146

3.5.3 Measures of Leverage and Influence 151

3.6 Variable Selection Methods in Regression 153

3.7 Generalized and Weighted Least Squares 160

3.7.1 Generalized Least Squares 160

3.7.2 Weighted Least Squares 163

3.7.3 Discounted Least Squares 168

3.8 Regression Models for General Time Series Data 184

3.8.1 Detecting Autocorrelation: The Durbin–Watson Test 186

3.8.2 Estimating the Parameters in Time Series Regression Models 191

3.9 Econometric Models 213

3.10 R Commands for Chapter 3 216

Exercises 226

4 Exponential Smoothing Methods 243

4.1 Introduction 243

4.2 First-Order Exponential Smoothing 249

4.2.1 The Initial Value, ỹ0 251

4.2.2 The Value of λ 251

4.3 Modeling Time Series Data 254

4.4 Second-Order Exponential Smoothing 257

4.5 Higher-Order Exponential Smoothing 269

4.6 Forecasting 270

4.6.1 Constant Process 270

4.6.2 Linear Trend Process 272

4.6.3 Estimation of σe2 284

4.6.4 Adaptive Updating of the Discount Factor 285

4.6.5 Model Assessment 287

4.7 Exponential Smoothing for Seasonal Data 288

4.7.1 Additive Seasonal Model 288

4.7.2 Multiplicative Seasonal Model 292

4.8 Exponential Smoothing of Biosurveillance Data 299

4.9 Exponential Smoothers and ARIMA Models 308

4.10 R Commands for Chapter 4 309

Exercises 321

5 Autoregressive Integrated Moving Average (ARIMA) Models 339

5.1 Introduction 339

5.2 Linear Models for Stationary Time Series 340

5.2.1 Stationarity 341

5.2.2 Stationary Time Series 341

5.3 Finite Order Moving Average Processes 345

5.3.1 The First-Order Moving Average Process, MA(1) 347

5.3.2 The Second-Order Moving Average Process, MA(2) 349

5.4 Finite Order Autoregressive Processes 350

5.4.1 First-Order Autoregressive Process, AR(1) 350

5.4.2 Second-Order Autoregressive Process, AR(2) 354

5.4.3 General Autoregressive Process, AR(p) 359

5.4.4 Partial Autocorrelation Function, PACF 360

5.5 Mixed Autoregressive–Moving Average Processes 367

5.5.1 Stationarity of ARMA(p, q) Process 368

5.5.2 Invertibility of ARMA(p, q) Process 368

5.5.3 ACF and PACF of ARMA(p, q) Process 369

5.6 Nonstationary Processes 376

5.6.1 Some Examples of ARIMA(p, d, q) Processes 377

5.7 Time Series Model Building 380

5.7.1 Model Identification 380

5.7.2 Parameter Estimation 381

5.7.3 Diagnostic Checking 382

5.7.4 Examples of Building ARIMA Models 383

5.8 Forecasting Arima Processes 392

5.9 Seasonal Processes 401

5.10 Arima Modeling of Biosurveillance Data 407

5.11 Final Comments 413

5.12 R Commands for Chapter 5 415

Exercises 427

6 Transfer Functions and Intervention Models 447

6.1 Introduction 447

6.2 Transfer Function Models 448

6.3 Transfer Function–Noise Models 456

6.4 Cross-Correlation Function 456

6.5 Model Specification 458

6.6 Forecasting with Transfer Function–Noise Models 476

6.7 Intervention Analysis 481

6.8 R Commands for Chapter 6 494

Exercises 508

7 Other Time Series Analysis and Forecasting Methods 517

7.1 Multivariate Time Series Models and Forecasting 517

7.1.1 Multivariate Stationary Process 518

7.1.2 Vector ARIMA Models 519

7.1.3 Vector AR (VAR) Models 520

7.2 State Space Models 526

7.3 Arch and Garch Models 531

7.4 Direct Forecasting of Percentiles 536

7.5 Combining Forecasts to Improve Prediction Performance 542

7.6 Aggregation and Disaggregation of Forecasts 547

7.7 Neural Networks and Forecasting 551

7.8 Spectral Analysis 559

7.9 Bayesian Methods in Forecasting 565

7.10 Some Comments on Practical Implementation and Use of Statistical Forecasting Procedures 572

7.11 R Commands for Chapter 7 576

Exercises 581

Appendix A Statistical Tables 595

Appendix B Data Sets for Exercises 615

Appendix C Introduction to R 683

Bibliography 689

Index 697