Multivariate Analysis with LISREL (eBook)

Karl G. Jöreskog is Professor Emeritus at Uppsala University, Sweden, and Senior Professor at the BI Norwegian School of Business in Oslo. He has received three honorary doctorates: from the Faculty of Economics and Statistics at the University of Padua, Italy, 1993, from the Norwegian School of Economics, Bergen, Norway, 1996, and from the Faculty of Psychology at the Friedrich-Schiller-Universität, Jena, Germany, 2004. Professor Jöreskog is a member of the Swedish Royal Academy of Sciences, a Fellow of the American Statistical Association, and an Honorary Fellow of the Royal Statistical Society. He has received many awards including the American Psychological Association Distinguished Award for the Applications of Psychology and the Psychometric Society Award for Career Achievement to Educational Measurement. Together with Dag Sörbom he developed the LISREL computer program.Ulf H. Olsson is Professor at Department of Economics and Provost at BI Norwegian Business School in Oslo with responsibility for research and academic resources. He has worked on structural equation modeling, statistical modeling and psychometrics and published several research articles in leading statistics and psychometric journals. Dr. Olsson has also authored textbooks on statistics and mathematics. In 2003 Olsson was awarded the BI Norwegian Business School’s research prize.Fan Y. Wallentin is Professor of Statistics at Uppsala University, Sweden. She received her Ph.D. in Statistics in 1997. She is a recipient of the Arnberg Prize from the Swedish Royal Academy of Sciences. Dr. Wallentin's program of research is on the theory and applications of latent variable modeling and other types of multivariate statistical analysis, particularly their applications in the social and behavioral sciences. She has published research articles in several leading statistics and psychometrics journals. She has taught courses on Structural Equation Models in Sweden, USA, China and several European countries. She has broad experience in statistical consultation for researchers in social and behavioral sciences.

Preface 6
Contents 8
About the Authors 15
1Getting Started 16
1.1 Importing Data 16
1.2 Graphs 19
1.3 Splitting the Data into Two Groups 24
1.4 Introduction to LISREL Syntaxes 26
1.5 Estimating Covariance or Correlation Matrices 30
1.6 Missing Values 33
1.7 Data Management 41
2Regression Models 49
2.1 Linear Regression 49
2.1.1 Estimation and Testing 51
2.1.2 Example: Cholesterol 53
2.1.3 Importing Data 53
2.1.4 Checking the Assumptions 59
2.1.5 The Effect of Increasing the Sample Size 66
2.1.6 Regression using Means, Variances, and Covariances 66
2.1.7 Standardized Solution 67
2.1.8 Predicting y When ln(y) is Used as the Dependent Variable 69
2.1.9 Example: Income 69
2.1.10 ANOVA and ANCOVA 72
2.1.11 Example: Biology 73
2.1.12 Conditional Regression 75
2.1.13 Example: Birthweight 75
2.1.14 Testing Equal Regressions 77
2.1.15 Example: Math on Reading by Career 78
2.1.16 Instrumental Variables and Two-Stage Least Squares 84
2.1.17 Example: Income and Money Supply 86
2.1.18 Example: Tintner’s Meat Market Model 89
2.1.19 Example: Klein’s Model I of US Economy 90
2.2 General Principles of SIMPLIS Syntax 93
2.2.1 Example: Income and Money Supply Using SIMPLIS Syntax 100
2.2.2 Example: Prediction of Grade Averages 102
2.2.3 Example: Prediction of Test Scores 104
2.2.4 Example: Union Sentiment of Textile Workers 106
2.3 The General Multivariate Linear Model 109
2.3.1 Introductory LISREL Syntax 111
2.3.2 Univariate Regression Model 112
2.3.3 Multivariate Linear Regression 115
2.3.4 Example: Prediction of Test Scores with LISREL Syntax 116
2.3.5 Recursive Systems 119
2.3.6 Example: Union Sentiment of Textile Workers with LISREL Syntax 119
2.3.7 Non-Recursive Systems 121
2.3.8 Example: Income and Money Supply with LISREL syntax 121
2.3.9 Direct, Indirect, and Total Effects 123
2.4 Logistic and Probit Regression 126
2.4.1 Continuous Predictors 126
2.4.2 Example: Credit Risk 127
2.4.3 Pseudo-R2s 129
2.4.4 Categorical Predictors 129
2.4.5 Example: Death Penalty Verdicts 130
2.4.6 Extensions of Logistic and Probit Regression 133
2.5 Censored Regression 133
2.5.1 Censored Normal Variables 134
2.5.2 Censored Normal Regression 136
2.5.3 Example: Affairs 137
2.5.4 Example: Reading and Spelling Tests 140
2.6 Multivariate Censored Regression 141
2.6.1 Example: Testscores 144
3Generalized Linear Models 148
3.1 Components of Generalized Linear Models 148
3.2 Exponential Family Distributions 149
3.2.1 Distributions and Link Functions 149
3.3 The Poisson-Log Model 150
3.3.1 Example: Smoking and Coronary Heart Disease 152
3.3.2 Example: Awards 157
3.4 The Binomial-Logit/Probit Model 161
3.4.1 Example: Death Penalty Verdicts Revisited 162
3.5 Log-linear Models 165
3.5.1 Example: Malignant Melanoma 166
3.6 Nominal Logistic Regression 169
3.6.1 Example: Program Choices 1 171
3.6.2 Example: Program Choices 2 175
3.7 Ordinal Logistic Regression 177
3.7.1 Example: Mental Health 178
3.7.2 Example: Car Preferences 180
4Multilevel Analysis 183
4.1 Basic Concepts and Issues in Multilevel Analysis 183
4.1.1 Multilevel Data and Multilevel Analysis 183
4.1.2 Examples of Multilevel Data 183
4.1.3 Terms Used for Two-level Models 184
4.1.4 Multilevel Analysis vs Linear Regression 184
4.1.5 Other Terminology 185
4.1.6 Populations and Subgroups 185
4.1.7 The Interaction Question 185
4.2 Within and Between Group Variation 186
4.2.1 Univariate Analysis 186
4.2.2 Example: Netherlands Schools, Univariate Case 186
4.2.3 Multivariate Analysis 193
4.2.4 Example: Netherlands Schools, Multivariate Case 193
4.3 The Basic Two-Level Model 195
4.3.1 Example: Math on Reading with Career-Revisited 197
4.4 Two-Level Model with Cross-Level Interaction 201
4.5 Likelihood, Deviance, and Chi-Square 202
4.5.1 Example: Math Achievement and Socioeconomic Status 203
4.6 Multilevel Analysis of Repeated Measurements 209
4.6.1 Example: Treatment of Prostate Cancer 210
4.6.2 Example: Learning Curves of Air Traffic Controllers 213
4.6.3 Example: Growth Curves for the Weight of Mice 220
4.6.4 Example: Growth Curves for Weight of Chicks on Four Diets 222
4.7 Multilevel Generalized Linear Models 229
4.7.1 Example: Social Mobility 229
4.8 The Basic Three-Level Model 235
4.8.1 Example: CPC Survey Data 236
4.9 Multivariate Multilevel Analysis 240
4.9.1 Example: Analysis of the Junior School Project Data (JSP) 242
5Principal Components (PCA) 248
5.1 Principal Components of a Covariance Matrix 248
5.1.1 Example: Five Meteorological Variables 252
5.2 Principal Components vs Factor Analysis 259
5.3 Principal Components of a Data Matrix 263
5.3.1 Example: PCA of Nine Psychological Variables 264
5.3.2 Example: Stock Market Prices 266
6Exploratory Factor Analysis (EFA) 268
6.1 The Factor Analysis Model and Its Estimation 269
6.2 A Population Example 276
6.2.1 Example: A Numeric Illustration 276
6.3 EFA with Continuous Variables 279
6.3.1 Example: EFA of Nine Psychological Variables (NPV) 279
6.4 EFA with Ordinal Varaibles 284
6.4.1 EFA of Binary Test Items 285
6.4.2 Example: Analysis of LSAT6 Items 285
6.4.3 EFA of Polytomous Tests and Survey Items 288
6.4.4 Example: Attitudes Toward Science and Technology 289
7Confirmatory Factor Analysis(CFA) 294
7.1 General Model Framework 295
7.2 Measurement Models 297
7.2.1 The Congeneric Measurement Model 297
7.2.2 Congeneric, parallel, and tau-equivalent measures 298
7.2.3 Example: Analysis of Reader Reliability in Essay Scoring 299
7.3 CFA with Continuous Variables 301
7.3.1 Continuous Variables without Missing Values 301
7.3.2 Example: CFA of Nine Psychological Variables 302
7.3.3 Estimating the Model by Maximum Likelihood 303
7.3.4 Analyzing Correlations 315
7.3.5 Continuous Variables with Missing Values 322
7.3.6 Example: Longitudinal Data on Math and English Scores 322
7.4 CFA with Ordinal Variables 329
7.4.1 Ordinal Variables without Missing Values 329
7.4.2 Ordinal Variables with Missing Values 339
7.4.3 Example: Measurement of Political Efficacy 340
8Structural Equation Models (SEM) with Latent Variables 351
8.1 Example: Hypothetical Model 351
8.1.1 Hypothetical Model with SIMPLIS Syntax 352
8.2 The General LISREL Model in LISREL Format 353
8.3 General Framework 354
8.3.1 Scaling of Latent Variables 355
8.3.2 Notation for LISREL Syntax 356
8.4 Special Cases of the General LISREL Model 357
8.4.1 Matrix Specification of the Hypothetical Model 357
8.4.2 LISREL syntax for the Hypothetical Model 359
8.5 Measurement Errors in Regression 360
8.5.1 Example: Verbal Ability in Grades 4 and 5 360
8.5.2 Example: Role Behavior of Farm Managers 361
8.6 Second-Order Factor Analysis 365
8.6.1 Example: Second-Order Factor of Nine Psychological Variables 367
8.7 Analysis of Correlation Structures 369
8.7.1 Example: CFA Model for NPV Estimated from Correlations 370
8.8 MIMIC Models 373
8.8.1 Example: Peer Influences and Ambition 373
8.8.2 Example: Continuous Causes and Ordinal Indicators 377
8.9 A Model for the Theory of Planned Behavior 381
8.9.1 Example: Attitudes to Drinking and Driving 381
8.10 Latent Variable Scores 384
8.10.1 Example: Panel Model for Political Democracy 384
9Analysis of Longitudinal Data 389
9.1 Two-wave Models 389
9.1.1 Example: Stability of Alienation 389
9.1.2 Example: Panel Model for Political Efficacy 394
9.2 Simplex Models 406
9.2.1 Example: A Simplex Model for Academic Performance 408
9.3 Latent Curve Models 409
9.3.1 Example: Treatment of Prostate Cancer 412
9.3.2 Example: Learning Curves for of Traffic Controllers 423
9.4 Latent Growth Curves and Dyadic Data 430
9.4.1 Example: Quality of Marriages 430
10Multiple Groups 437
10.1 Factorial Invariance 437
10.2 Multiple Groups with Continuous Variables 439
10.2.1 Equal Regressions 439
10.2.2 Example: STEP Reading and Writing Tests in Grades 5 and 7 439
10.2.3 Estimating Means of Latent Variables 442
10.2.4 Confirmatory Factor Analysis with Multiple Groups 446
10.2.5 Example: Chicago Schools Data 446
10.2.6 MIMIC Models for Multiple Groups 449
10.2.7 Twin Data Models 454
10.2.8 Example: Heredity of BMI 457
10.3 Multiple Groups with Ordinal Variables 464
10.3.1 Example: The Political Action Survey 464
10.3.2 Data Screening 465
10.3.3 Multigroup Models 468
11Appendix A: Basic Matrix Algebra and Statistics 478
11.1 Basic Matrix Algebra 478
11.2 Basic Statistical Concepts 486
11.3 Basic Multivariate Statistics 488
11.4 Measurement Scales 489
12Appendix B: Testing Normality 490
12.1 Univariate Skewness and Kurtosis 490
12.2 Multivariate Skewness and Kurtosis 493
13Appendix C: Computational Notes on Censored Regression 495
13.1 Computational Notes on Univariate Censored Regression 495
13.2 Computational Notes on Multivariate Censored Regression 497
14Appendix D: Normal Scores 499
15Appendix E: Asessment of Fit 500
15.1 From Theory to Statistical Model 500
15.2 Nature of Inference 502
15.3 Three Situations 502
15.4 Selection of One of Several Specified Models 504
15.5 Model Assessment and Modification 505
15.6 Chi-squares 506
15.7 Goodness-of-Fit Indices 507
15.8 Population Error of Approximation 507
15.9 Other Fit Indices 508
16Appendix F: General Statistical Theory 510
16.1 Continuous Variables 510
16.1.1 Data and Sample Statistics 510
16.1.2 The Multivariate Normal Distribution 510
16.1.3 The Multivariate Normal Likelihood 511
16.1.4 Likelihood, Deviance, and Chi-square 513
16.1.5 General Covariance Structures 514
16.1.6 The Independence Model 518
16.1.7 Mean and Covariance Structures 518
16.1.8 Augmented Moment Matrix 520
16.1.9 Multiple Groups 520
16.1.10 Maximum Likelihood with Missing Values (FIML) 522
16.1.11 Multiple Imputation 523
16.2 Ordinal Variables 523
16.2.1 Estimation by FIML 524
16.2.2 Estimation via Polychorics 526
17Appendix G: Iteration Algorithms 529
17.1 General Definitions 529
17.2 Technical Parameters 530
17.3 The Davidon-Fletcher-Powell Method 532
17.4 Convergence Criterion 532
17.5 Line Search 532
17.6 Interpolation and Extrapolation Formulas 538
Bibliography 540
Subject Index 555