Developing Econometrics

Hengqing Tong, Department of Mathematics, Wuhan University of Technology, P.R.China T. Krishna Kumar, Indian Institute of Management, Samkhya Analytica India Private Limited, Bangalore, India Yangxin Huang, Department of Epidemiology and Biostatistics, University of South Florida, USA

Foreword xi Preface xiii

Acknowledgements xvii

1 Introduction 1

1.1 Nature and Scope of Econometrics 2

1.1.1 What is Econometrics and Why Study Econometrics? 2

1.1.2 Econometrics and Scientific Credibility of Business and Economic Decisions 4

1.2 Types of Economic Problems, Types of Data, and Types of Models 5

1.2.1 Experimental Data from a Marketing Experiment 5

1.2.2 Cross-Section Data: National Sample Survey Data on Consumer Expenditure 6

1.2.3 Non-Experimental Data Taken from Secondary Sources: The Case of Pharmaceutical Industry in India 8

1.2.4 Loan Default Risk of a Customer and the Problem Facing Decision on a Loan Application 9

1.2.5 Panel Data: Performance of Banks in India by the Type of Ownership after Economic Reforms 10

1.2.6 Single Time Series Data: The Bombay Stock Exchange (BSE) Index 12

1.2.7 Multiple Time Series Data: Stock Prices in BRIC Countries 12

1.3 Pattern Recognition and Exploratory Data Analysis 14

1.3.1 Some Basic Issues in Econometric Modeling 14

1.3.2 Exploratory Data Analysis Using Correlations and Scatter Diagrams: The Relative Importance of Managerial Function and Labor 16

1.3.3 Cleaning and Reprocessing Data to Discover Patterns: BSE Index Data 22

1.4 Econometric Modeling: The Roadmap of This Book 24

1.4.1 The Econometric Modeling Strategy 24

1.4.2 Plan of the Book 25

Electronic References for Chapter 1 27

References 27

2 Independent Variables in Linear Regression Models 29

2.1 Brief Review of Linear Regression 29

2.1.1 Brief Review of Univariate Linear Regression 29

2.1.2 Brief Review of Multivariate Linear Regression 38

2.2 Selection of Independent Variable and Stepwise Regression 49

2.2.1 Principles of Selection of Independent Variables 49

2.2.2 Stepwise Regression 52

2.3 Multivariate Data Transformation and Polynomial Regression 57

2.3.1 Linear Regression after Multivariate Data Transformation 57

2.3.2 Polynomial Regression on an Independent Variable 61

2.3.3 Multivariable Polynomial Regression 62

2.4 Column Multicollinearity in Design Matrix and Ridge Regression 65

2.4.1 Effect of Column Multicollinearity of Design Matrix 65

2.4.2 Ridge Regression 68

2.4.3 Ridge Trace Analysis and Ridge Parameter Selection 70

2.4.4 Generalized Ridge Regression 71

2.5 Recombination of Independent Variable and Principal Components Regression 72

2.5.1 Concept of Principal Components Regression 72

2.5.2 Determination of Principal Component 74

Electronic References for Chapter 2 79

References 80

3 Alternative Structures of Residual Error in Linear Regression Models 83

3.1 Heteroscedasticity: Consequences and Tests for Its Existence 85

3.1.1 Consequences of Heteroscedasticity 85

3.1.2 Tests for Heteroscedasticity 87

3.2 Generalized Linear Model with Covariance Being a Diagonal Matrix 90

3.2.1 Diagonal Covariance Matrix and Weighted Least Squares 90

3.2.2 Model with Two Unknown Variances 91

3.2.3 Multiplicative Heteroscedastic Model 92

3.3 Autocorrelation in a Linear Model 95

3.3.1 Linear Model with First-Order Residual Autoregression 96

3.3.2 Autoregressive Conditional Heteroscedasticity (ARCH) Model 101

3.4 Generalized Linear Model with Positive Definite Covariance Matrix 106

3.4.1 Model Definition, Parameter Estimation and Hypothesis Tests 106

3.4.2 Some Equivalent Conditions 108

3.5 Random Effects and Variance Component Model 109

3.5.1 Random Effect Regression Model 109

3.5.2 The Variance Component Model 112

3.5.3 Analysis of Variance Method to Solve Variance Component Model 113

3.5.4 Minimum Norm Quadratic Unbiased Estimation (MINQUE) to Solve Variance Component 121

3.5.5 Maximum Likelihood Method to Solve Variance Component Model 124

Electronic References for Chapter 3 125

References 125

4 Discrete Variables and Nonlinear Regression Model 129

4.1 Regression Model When Independent Variables are Categorical 130

4.1.1 Problem About Wage and Gender Differences 131

4.1.2 Structural Changes in the Savings Function (Use of Categorical Variables in Combination with Continuous Variables) 133

4.1.3 Cross Section Analysis 138

4.1.4 Seasonal Analysis Model 141

4.2 Models with Categorical or Discrete Dependent Variables 144

4.2.1 Linear Model with Binary Dependent Variable 144

4.2.2 Logit Regression Model 148

4.2.3 Probit Regression Model 153

4.2.4 Tobit Regression Model 154

4.3 Nonlinear Regression Model and Its Algorithm 160

4.3.1 The Least Squares Estimate for Nonlinear Regression Model 162

4.3.2 Maximum Likelihood Estimation of Nonlinear Regression Model 164

4.3.3 Equivalence of Maximum Likelihood Estimation and Least Squares Estimation 166

4.4 Nonlinear Regression Models in Practice 169

4.4.1 Growth Curve Models 169

4.4.2 Box–Cox Transformation Model 176

4.4.3 Survival Data and Failure Rate Model 177

4.4.4 Total Factor Productivity (TFP) 181

Electronic References for Chapter 4 188

References 188

5 Nonparametric and Semiparametric Regression Models 193

5.1 Nonparametric Regression and Weight Function Method 194

5.1.1 The Concept of Nonparametric Regression 194

5.1.2 Weight Function Method 196

5.2 Semiparametric Regression Model 199

5.2.1 Linear Semiparametric Regression Model 202

5.2.2 Single-Index Semiparametric Regression Model 205

5.3 Stochastic Frontier Regression Model 208

5.3.1 Stochastic Frontier Linear Regression Model and Asymptotically Efficient Estimator of Its Parameters 208

5.3.2 Semiparametric Stochastic Frontier Model 210

Electronic References for Chapter 5 212

References 213

6 Simultaneous Equations Models and Distributed Lag Models 215

6.1 Simultaneous Equations Models and Inconsistency of OLS Estimators 216

6.1.1 Demand-and-Supply Model, Keynesian Model and Wage-Price Model (Phillips Curve) 218

6.1.2 Macroeconomic IS Model, LM Model and Klein’s Econometric Model 220

6.1.3 Inconsistency of OLS Estimation 222

6.2 Statistical Inference for Simultaneous Equations Models 223

6.2.1 Indirect Least Squares and Generalized Least Squares 224

6.2.2 Two Stage Least Squares 229

6.3 The Concepts of Lag Regression Models 235

6.3.1 Consumption Lag 236

6.3.2 Inflation Lag 237

6.3.3 Deposit Re-Creation 238

6.4 Finite Distributed Lag Models 239

6.4.1 Estimation of Distributed Lag Models When the Lag Length is Known and Finite 239

6.4.2 The Determination of Distributed Lag Length 239

6.5 Infinite Distributed Lag Models 242

6.5.1 Adaptive Expectations Model and Partial Adjustment Model 243

6.5.2 Koyck Transformation and Estimation of Geometric Lag Models 245

Electronic References for Chapter 6 249

References 250

7 Stationary Time Series Models 253

7.1 Auto-Regression Model AR( p) 255

7.1.1 AR( p) Model and Stationarity 255

7.1.2 Auto-Covariance Function and Autocorrelation Function of AR( p) Model 258

7.1.3 Spectral Density of AR( p) Model and Partial Correlation Coefficient 263

7.1.4 Estimation of Parameters for AR( p) Model with Known Order p 267

7.1.5 Order Identification for AR( p) Process 274

7.2 Moving Average Model MA(q) 276

7.2.1 MA(q) Model and Its Properties 276

7.2.2 Parameter Estimation of MA(q) Model When the Order q is Known 278

7.2.3 Spectral Density Estimation for MA(q) Process 282

7.2.4 Order Identification for MA(q) Process 284

7.3 Auto-Regressive Moving-Average Process ARMA( p, q) 285

7.3.1 ARMA(p, q) Model and Its Properties 285

7.3.2 Parameter Estimations for ARMA(p, q) Model 288

7.3.3 Test for ARMA( p, q) Model 291

7.3.4 Order Identification for ARMA( p, q) Model 291

7.3.5 Univariate Time Series Modeling: The Basic Issues and Approaches 292

Electronic References for Chapter 7 293

References 293

8 Multivariate and Nonstationary Time Series Models 297

8.1 Multivariate Stationary Time Series Model 299

8.1.1 General Description of Multivariable Stationary Time Series Model 299

8.1.2 Estimation of Mean and Autocovariance Function of Multivariate Stationary Time Series 300

8.1.3 Vector Autoregression Model of Order p: VAR( p) 301

8.1.4 Wold Decomposition and Impulse-Response 301

8.1.5 Variance Decomposition with VAR( p) 306

8.1.6 Granger Causality with VAR(p) Specification 309

8.2 Nonstationary Time Series 311

8.2.1 Stochastic Trends and Unit Root Processes 311

8.2.2 Test for Unit Root Hypothesis 314

8.3 Cointegration and Error Correction 321

8.3.1 The Concept and Representation of Cointegration 322

8.3.2 Simultaneous (Structural) Equation System (SES) and Vector Auto Regression (VAR) 324

8.3.3 Cointegration and Error Correction Representation 325

8.3.4 Estimation of Parameters of Cointegration Process 329

8.3.5 Test of Hypotheses on the Number of Cointegrating Equations 330

8.4 Autoregression Conditional Heteroscedasticity in Time Series 333

8.4.1 ARCH Model 334

8.4.2 Generalized ARCH Model—GARCH Model 338

8.4.3 Other Generalized Forms of ARCH Model 342

8.5 Mixed Models of Multivariate Regression with Time Series 346

8.5.1 Mixed Model of Multivariate Regression with Time Series 346

8.5.2 Mixed Model of Multivariate Regression and Cointegration with Time Series 349

Electronic References for Chapter 8 353

References 353

9 Multivariate Statistical Analysis and Data Analysis 357

9.1 Model of Analysis of Variance 358

9.1.1 Single Factor Analysis of Variance Model 358

9.1.2 Two Factor Analysis of Variance with Non-Repeated Experiment 361

9.1.3 Two Factor Analysis of Variance with Repeated Experiment 364

9.2 Other Multivariate Statistical Analysis Models 370

9.2.1 Discriminate Analysis Model 370

9.2.2 Factor Analysis Model 376

9.2.3 Principal Component Analysis and Multidimensional Scaling Method 380

9.2.4 Canonical Correlation Analysis 384

9.3 Customer Satisfaction Model and Path Analysis 387

9.3.1 Customer Satisfaction Model and Structural Equations Model 387

9.3.2 Partial Least Square and the Best Iterative Initial Value 391

9.3.3 Definite Linear Algorithm for SEM 399

9.3.4 Multi-Layers Path Analysis Model 402

9.4 Data Analysis and Process 404

9.4.1 Panel Data Analysis 404

9.4.2 Truncated Data Analysis 405

9.4.3 Censored Data Analysis 406

9.4.4 Duration Data Analysis 407

9.4.5 High Dimensional Data Visualization 409

Electronic References for Chapter 9 412

References 413

10 Summary and Further Discussion 415

10.1 About Probability Distributions: Parametric and Non-Parametric 416

10.1.1 Distributions of Functions of Random Variables 416

10.1.2 Parametric, Non-Parametric, and Semi-Parametric Specification of Distributions 417

10.1.3 Non-Parametric Specification of Density Functions 418

10.2 Regression 421

10.2.1 Regression as Conditional Mean of the Dependent Variable 421

10.2.2 Regressions with Homoscedastic and Heteroscedastic Variance 421

10.2.3 General Regression Functions: Quantiles and Quantile Regression 423

10.2.4 Design of Experiments, Regression, and Analysis of Variance 424

10.3 Model Specification and Prior Information 425

10.3.1 Data Generation Process (DGP) and Economic Structure 426

10.3.2 Deterministic but Unknown Parameters and Model Specification as a Maintained Hypothesis 428

10.3.3 Stochastic Prior Information on Unknown Parameters 429

10.4 Classical Theory of Statistical Inference 430

10.4.1 The Likelihood Function, Sufficient Statistics, Complete Statistics, and Ancillary Statistics 430

10.4.2 Different Methods of Estimation of Unknown Parameters 434

10.4.3 Biased and Unbiased Estimators, Consistency of Estimators 437

10.4.4 Information Limit to Variance of an Estimator, Cramer-Rao Bound, and Rao-Blackwell Theorem 438

10.4.5 Approximate Sufficiency and Robust Estimation 440

10.5 Computation of Maximum Likelihood Estimates 441

10.5.1 Newton-Raphson Method and Rao’s Method of Scoring 442

10.5.2 Davidon-Fletcher-Powell-Reeves Conjugate Gradient Procedure 443

10.5.3 Estimates of the Variance Covariance Matrix of Maximum Likelihood Estimators 444

10.6 Specification Searches 445

10.6.1 Choice Between Alternate Specifications: Akaike and Schwarz Information Criteria 445

10.6.2 Generalized Information and Complexity-Based Model Choice Criterion 447

10.6.3 An Illustration of Model Choice: Engel Curve for Food Consumption in India 448

10.7 Resampling and Sampling Distributions – The Bootstraps Method 450

10.7.1 The Concept of Resampling and the Bootstraps Method 450

10.7.2 Bootstraps in Regression Models 452

10.8 Bayesian Inference 454

10.8.1 The Bayes Rule 454

10.8.2 Choice of Prior Probability Distribution for the Parameter 455

10.8.3 Bayesian Concepts for Statistical Inference 456

Electronic References for Chapter 10 457

References 458

Index 461