Time Series Analysis

The late George E. P. Box, PhD, was professor emeritus of statistics at the University of Wisconsin-Madison. He was a Fellow of the American Academy of Arts and Sciences and a recipient of the Samuel S. Wilks Memorial Medal of the American Statistical Association, the Shewhart Medal of the American Society for Quality, and the Guy Medal in Gold of the Royal Statistical Society. Dr. Box was also author of seven Wiley books. The late Gwilym M. Jenkins, PhD, was professor of systems engineering at Lancaster University in the United Kingdom, where he was also founder and managing director of the International Systems Corporation of Lancaster. A Fellow of the Institute of Mathematical Statistics and the Institute of Statisticians, Dr. Jenkins had a prestigious career in both academia and consulting work that included positions at Imperial College London, Stanford University, Princeton University, and the University of Wisconsin-Madison. He was widely known for his work on time series analysis, most notably his groundbreaking work with Dr. Box on the Box-Jenkins models. The late Gregory C. Reinsel, PhD, was professor and former chair of the department of Statistics at the University of Wisconsin-Madison. Dr. Reinsel's expertise was focused on time series analysis and its applications in areas as diverse as economics, ecology, engineering, and meteorology. He authored over seventy refereed articles and three books, and was a Fellow of both the American Statistical Association and the Institute of Mathematical Statistics. Greta M. Ljung, PhD, is a statistical consultant residing in Lexington, MA. She received her doctorate from the University of Wisconsin-Madison where she did her research in time series analysis under the direction of Professor George Box. Dr. Ljung's career includes teaching positions at Boston University and Massachusetts Institute of Technology, and a position as Principal Scientist at AIR Worldwide in Boston. Her many accomplishments include joint work with George Box on a time series goodness of fit test, which is widely applied in econometrics and other fields.

Preface to the Fifth Edition xix

Preface to the Fourth Edition xxiii

Preface to the Third Edition xxv

1 Introduction 1

1.1 Five Important Practical Problems 2

1.2 Stochastic and Deterministic Dynamic Mathematical Models 6

1.3 Basic Ideas in Model Building 14

Appendix A. 1 Use of the R Software 17

Exercises 18

Part One Stochastic Models and Their Forecasting 19

2 Autocorrelation Function and Spectrum of Stationary Processes 21

2.1 Autocorrelation Properties of Stationary Models 21

2.2 Spectral Properties of Stationary Models 34

Appendix A2. 1 Link Between the Sample Spectrum and Autocovariance Function Estimate 43

Exercises 44

3 Linear Stationary Models 47

3.1 General Linear Process 47

3.2 Autoregressive Processes 54

3.3 Moving Average Processes 68

3.4 Mixed Autoregressive--Moving Average Processes 75

Appendix A3. 1 Autocovariances Autocovariance Generating Function, and Stationarity Conditions for a General Linear Process 82

Appendix A3. 2 Recursive Method for Calculating Estimates of Autoregressive Parameters 84

Exercises 86

4 Linear Nonstationary Models 88

4.1 Autoregressive Integrated Moving Average Processes 88

4.2 Three Explicit Forms for the ARIMA Model 97

4.3 Integrated Moving Average Processes 106

Appendix A4. 1 Linear Difference Equations 116

Appendix A4. 2 IMA(0, 1, 1) Process with Deterministic Drift 121

Appendix A4. 3 ARIMA Processes with Added Noise 122

Exercises 126

5 Forecasting 129

5.1 Minimum Mean Square Error Forecasts and Their Properties 129

5.2 Calculating Forecasts and Probability Limits 135

5.3 Forecast Function and Forecast Weights 139

5.4 Examples of Forecast Functions and Their Updating 144

5.5 Use of State-Space Model Formulation for Exact Forecasting 155

5.6 Summary 162

Appendix A5. 1 Correlation Between Forecast Errors 164

Appendix A5. 2 Forecast Weights for any Lead Time 166

Appendix A5. 3 Forecasting in Terms of the General Integrated Form 168

Exercises 174

Part Two STOCHASTIC MODEL BUILDING 177

6 Model Identification 179

6.1 Objectives of Identification 179

6.2 Identification Techniques 180

6.3 Initial Estimates for the Parameters 194

6.4 Model Multiplicity 202

Appendix A6. 1 Expected Behavior of the Estimated Autocorrelation Function for a Nonstationary Process 206

Exercises 207

7 Parameter Estimation 209

7.1 Study of the Likelihood and Sum-of-Squares Functions 209

7.2 Nonlinear Estimation 226

7.3 Some Estimation Results for Specific Models 236

7.4 Likelihood Function Based on the State-Space Model 242

7.5 Estimation Using Bayes’ Theorem 245

Appendix A7. 1 Review of Normal Distribution Theory 251

Appendix A7. 2 Review of Linear Least-Squares Theory 256

Appendix A7. 3 Exact Likelihood Function for Moving Average and Mixed Processes 259

Appendix A7. 4 Exact Likelihood Function for an Autoregressive Process 266

Appendix A7. 5 Asymptotic Distribution of Estimators for Autoregressive Models 274

Appendix A7. 6 Examples of the Effect of Parameter Estimation Errors on Variances of Forecast Errors and Probability Limits for Forecasts 277

Appendix A. 7 Special Note on Estimation of Moving Average Parameters 280

Exercises 280

8 Model Diagnostic Checking 284

8.1 Checking the Stochastic Model 284

8.2 Diagnostic Checks Applied to Residuals 287

8.3 Use of Residuals to Modify the Model 301

Exercises 303

9 Analysis of Seasonal Time Series 305

9.1 Parsimonious Models for Seasonal Time Series 305

9.2 Representation of the Airline Data by a Multiplicative (0, 1, 1) × (0, 1, 1) 12 Model 310

9.3 Some Aspects of More General Seasonal ARIMA Models 325

9.4 Structural Component Models and Deterministic Seasonal Components 331

9.5 Regression Models with Time Series Error Terms 339

Appendix A9. 1 Autocovariances for Some Seasonal Models 345

Exercises 349

10 Additional Topics and Extensions 352

10.1 Tests for Unit Roots in ARIMA Models 353

10.2 Conditional Heteroscedastic Models 361

10.3 Nonlinear Time Series Models 377

10.4 Long Memory Time Series Processes 385

Exercises 392

Part Three Transfer Function and Multivariate Model Building 395

11 Transfer Function Models 397

11.1 Linear Transfer Function Models 397

11.2 Discrete Dynamic Models Represented by Difference Equations 404

11.3 Relation Between Discrete and Continuous Models 414

Appendix A11. 1 Continuous Models with Pulsed Inputs 420

Appendix A11. 2 Nonlinear Transfer Functions and Linearization 424

Exercises 426

12 Identification, Fitting, and Checking of Transfer Function Models 428

12.1 Cross-Correlation Function 429

12.2 Identification of Transfer Function Models 435

12.3 Fitting and Checking Transfer Function Models 446

12.4 Some Examples of Fitting and Checking Transfer Function Models 453

12.5 Forecasting with Transfer Function Models Using Leading Indicators 461

12.6 Some Aspects of the Design of Experiments to Estimate Transfer Functions 469

Appendix A12.1 Use of Cross-Spectral Analysis for Transfer Function Model Identification 471

Appendix A12.2 Choice of Input to Provide Optimal Parameter Estimates 473

Exercises 477

13 Intervention Analysis, Outlier Detection, and Missing Values 481

13.1 Intervention Analysis Methods 481

13.2 Outlier Analysis for Time Series 488

13.3 Estimation for ARMA Models with Missing Values 495

Exercises 502

14 Multivariate Time Series Analysis 505

14.1 Stationary Multivariate Time Series 506

14.2 Vector Autoregressive Models 509

14.3 Vector Moving Average Models 524

14.4 Vector Autoregressive--Moving Average Models 527

14.5 Forecasting for Vector Autoregressive--Moving Average Processes 534

14.6 State-Space Form of the VARMA Model 536

14.7 Further Discussion of VARMA Model Specification 539

14.8 Nonstationarity and Cointegration 546

Appendix A14. 1 Spectral Characteristics and Linear Filtering Relations for Stationary Multivariate Processes 552

Exercises 554

Part Four Design of Discrete Control Schemes 559

15 Aspects of Process Control 561

15.1 Process Monitoring and Process Adjustment 562

15.2 Process Adjustment Using Feedback Control 566

15.3 Excessive Adjustment Sometimes Required by MMSE Control 580

15.4 Minimum Cost Control with Fixed Costs of Adjustment and Monitoring 582

15.5 Feedforward Control 588

15.6 Monitoring Values of Parameters of Forecasting and Feedback Adjustment Schemes, 599

Appendix A5. 1 Feedback Control Schemes Where the Adjustment Variance Is Restricted, 600

Appendix A15. 2 Choice of the Sampling Interval 609

Exercises 613

Part Five Charts and Tables 617

Collection of Tables and Charts 619

Collection of Time Series Used for Examples in the Text and in Exercises 625

References 642

Index 659