Methods of Multivariate Analysis 3e

ALVIN C. RENCHER is Professor Emeritus in the Department of Statistics at Brigham Young University. A Fellow of the American Statistical Association, he is the author of Linear Models in Statistics, Second Edition and Multivariate Statistical Inference and Applications, both published by Wiley. WILLIAM F. CHRISTENSEN is Professor in the Department of Statistics at Brigham Young University. He has been published extensively in his areas of research interest, which include multivariate analysis, resampling methods, and spatial and environmental statistics.

Preface xvii Acknowledgments xxi 1 Introduction 1 1.1 Why Multivariate Analysis? 1 1.2 Prerequisites 3 1.3 Objectives 3 1.4 Basic Types of Data And Analysis 4 2 Matrix Algebra 7 2.1 Introduction 7 2.2 Notation and Basic Definitions 8 2.3 Operations 11 2.4 Partitioned Matrices 22 2.5 Rank 23 2.6 Inverse 25 2.7 Positive Definite Matrices 26 2.8 Determinants 28 2.9 Trace 31 2.10 Orthogonal Vectors and Matrices 31 2.11 Eigenvalues and Eigenvectors 32 2.12 Kronecker and VEC Notation 37 Problems 39 3 Characterizing and Displaying Multivariate Data 47 3.1 Mean and Variance of a Univariate Random Variable 47 3.2 Covariance and Correlation Of Bivariate Random Variables 49 3.3 Scatter Plots of Bivariate Samples 55 3.4 Graphical Displays for Multivariate Samples 56 3.5 Dynamic Graphics 58 3.6 Mean Vectors 63 3.7 Covariance Matrices 66 3.8 Correlation Matrices 69 3.9 Mean Vectors and Covariance Matrices for Subsets of Variables 71 3.9.1 Two Subsets 71 3.9.2 Three or More Subsets 73 3.10 Linear Combinations of Variables 75 3.10.1 Sample Properties 75 3.10.2 Population Properties 81 3.11 Measures of Overall Variability 81 3.12 Estimation of Missing Values 82 3.13 Distance Between Vectors 84 Problems 85 4 The Multivariate Normal Distribution 91 4.1 Multivariate Normal Density Function 91 4.2 Properties of Multivariate Normal Random Variables 94 4.3 Estimation in the Multivariate Normal 99 4.4 Assessing Multivariate Normality 101 4.5 Transformations to Normality 108 4.6 Outliers 111 Problems 117 5 Tests on One or Two Mean Vectors 125 5.1 Multivariate Versus Univariate Tests 125 5.2 Tests on With ??Known 126 5.3 Tests on When ??is Unknown 130 5.4 Comparing two Mean Vectors 134 5.5 Tests on Individual Variables Conditional on Rejection of H0 by the T2-test 139 5.6 Computation of T2 143 5.7 Paired Observations Test 145 5.8 Test for Additional Information 149 5.9 Profile Analysis 152 Profile Analysis 154 Problems 161 6 Multivariate Analysis of Variance 169 6.1 One-way Models 169 6.2 Comparison of the Four Manova Test Statistics 189 6.3 Contrasts 191 6.4 Tests on Individual Variables Following Rejection of H0 by the Overall Manova Test 195 6.5 Two-Way Classification 198 6.6 Other Models 207 6.7 Checking on the Assumptions 210 6.8 Profile Analysis 211 6.9 Repeated Measures Designs 215 6.10 Growth Curves 232 6.11 Tests on a Subvector 241 Problems 244 7 Tests on Covariance Matrices 259 7.1 Introduction 259 7.2 Testing a Specified Pattern for 259 7.3 Tests Comparing Covariance Matrices 265 7.4 Tests of Independence 269 Problems 276 8 Discriminant Analysis: Description of Group Separation 281 8.1 Introduction 281 8.2 The Discriminant Function for two Groups 282 8.3 Relationship Between two-group Discriminant Analysis and Multiple Regression 286 8.4 Discriminant Analysis for Several Groups 288 8.5 Standardized Discriminant Functions 292 8.6 Tests of Significance 294 8.7 Interpretation of Discriminant Functions 298 8.8 Scatter Plots 301 8.9 Stepwise Selection of Variables 303 Problems 306 9 Classification Analysis: Allocation of Observations to Groups309 9.1 Introduction 309 9.2 Classification into two Groups 310 9.3 Classification into Several Groups 314 9.4 Estimating Misclassification Rates 318 9.5 Improved Estimates of Error Rates 320 9.6 Subset Selection 322 9.7 Nonparametric Procedures 326 Problems 336 10 Multivariate Regression 339 10.1 Introduction 339 10.2 Multiple Regression: Fixed X's 340 10.3 Multiple Regression: Random X's 354 10.4 Multivariate Multiple Regression: Estimation 354 10.5 Multivariate Multiple Regression: Hypothesis Tests 364 10.6 Multivariate Multiple Regression: Prediction 370 10.7 Measures of Association Between the Y's and the X's 372 10.8 Subset Selection 374 10.9 Multivariate Regression: Random X's 380 Problems 381 11 Canonical Correlation 385 11.1 Introduction 385 11.2 Canonical Correlations and Canonical Variates 385 11.3 Properties of Canonical Correlations 390 11.4 Tests of Significance 391 11.5 Interpretation 395 11.6 Relationships of Canonical Correlation Analysis to Other Multivariate Problems 402 12 Principal Component Analysis 405 12.1 Introduction 405 12.2 Geometric and Algebraic Bases of Principal Components 406 12.3 Principal Components and Perpendicular Regression 412 12.4 Plotting of Principal Components 414 12.5 Principal Components from the Correlation Matrix 419 12.6 Deciding How Many Components to Retain 423 12.7 Information in the Last Few Principal Components 427 12.8 Interpretation of Principal Components 427 12.9 Selection of Variables 430 Problems 432 13 Exploratory Factor Analysis 435 13.1 Introduction 435 13.2 Orthogonal Factor Model 437 13.3 Estimation of Loadings and Communalities 442 13.4 Choosing the Number of Factors, m 453 13.5 Rotation 457 13.6 Factor Scores 466 13.7 Validity of the Factor Analysis Model 470 13.8 Relationship of Factor Analysis to Principal Component Analysis 475 Problems 476 14 Confirmatory Factor Analysis 479 14.1 Introduction 479 14.2 Model Specification and Identification 480 14.3 Parameter Estimation and Model Assessment 487 14.4 Inference for Model Parameters 492 14.5 Factor Scores 495 Problems 496 15 Cluster Analysis 501 15.1 Introduction 501 15.2 Measures of Similarity or Dissimilarity 502 15.3 Hierarchical Clustering 505 15.4 Nonhierarchical Methods 531 15.5 Choosing the Number of Clusters 544 15.6 Cluster Validity 546 15.7 Clustering Variables 547 Problems 548 16 Graphical Procedures 555 16.1 Multidimensional Scaling 555 16.2 Correspondence Analysis 565 16.3 Biplots 580 Problems 588 Appendix A: Tables 597 Appendix B: Answers and Hints to Problems 637 Appendix C: Data Sets and SAS Files 727 References 729 Index 747