Multivariate Statistics – High–Dimensional and e–Sample Approximations

Yasunori Fujikoshi, DSc, is Professor Emeritus at Hiroshima University (Japan) and Visiting Professor in the Department of Mathematics at Chuo University (Japan). He has authored over 150 journal articles in the area of multivariate analysis. Vladimir V. Ulyanov, DSc, is Professor in the Department of Mathematical Statistics at Moscow State University (Russia) and is the author of nearly fifty journal articles in his areas of research interest, which include weak limit theorems, probability measures on topological spaces, and Gaussian processes. Ryoichi Shimizu, DSc, is Professor Emeritus at the Institute of Statistical Mathematics (Japan) and is the author of numerous journal articles on probability distributions.

Preface. Glossary of Notation and Abbreviations. 1 Multivariate Normal and Related Distributions. 1.1 Random Vectors. 1.1.1 Mean Vector and Covariance Matrix. 1.1.2 Characteristic Function and Distribution. 1.2 Multivariate Normal Distribution. 1.2.1 Bivariate Normal Distribution. 1.2.2 Definition. 1.2.3 Some Properties. 1.3 Spherical and Elliptical Distributions. 1.4 Multivariate Cumulants. Problems. 2 Wishart Distribution. 2.1 Definition. 2.2 Some Basic Properties. 2.3 Functions of Wishart Matrices. 2.4 Cochran's Theorem. 2.5 Asymptotic Distributions. Problems. 3 Hotelling's T 2 and Lambda Statistics. 3.1 Hotelling's T 2 and Lambda Statistics. 3.1.1 Distribution of the T 2 Statistic. 3.1.2 Decomposition of T 2 and D 2 . 3.2 Lambda-Statistic. 3.2.1 Motivation of Lambda Statistic. 3.2.2 Distribution of Lambda Statistic. 3.3 Test for Additional Information. 3.3.1 Decomposition of Lambda Statistic. Problems. 4 Correlation Coefficients. 4.1 Ordinary Correlation Coefficients. 4.1.1 Population Correlation. 4.1.2 Sample Correlation. 4.2 Multiple Correlation Coefficient. 4.2.1 Population Multiple Correlation. 4.2.2 Sample Multiple Correlation. 4.3 Partial Correlation. 4.3.1 Population Partial Correlation. 4.3.2 Sample Partial Correlation. 4.3.3 Covariance Selection Model. Problems. 5 Asymptotic Expansions for Multivariate Basic Statistics. 5.1 Edgeworth Expansion and its Validity. 5.2 The Sample Mean Vector and Covariance Matrix. 5.3 T 2 Statistic. 5.3.1 Outlines of Two Methods. 5.3.2 Multivariate t-Statistic. 5.3.3 Asymptotic Expansions. 5.4 Statistics with a Class of Moments. 5.4.1 Large-Sample Expansions. 5.4.2 High-Dimensional Expansions. 5.5 Perturbation Method. 5.6 Cornish-Fisher Expansions. 5.6.1 Expansion Formulas. 5.6.2 Validity of Cornish-Fisher Expansions. 5.7 Transformations for Improved Approximations. 5.8 Bootstrap Approximations. 5.9 High-Dimensional Approximations. 5.9.1 Limiting Spectral Distribution. 5.9.2 Central Limit Theorem. 5.9.3 Martingale Limit Theorem. 5.9.4 Geometric Representation. Problems. 6 MANOVA Models. 6.1 Multivariate One-Way Analysis of Variance. 6.2 Multivariate Two-Way Analysis of Variance. 6.3 MANOVA Tests. 6.3.1 Test Criteria. 6.3.2 Large-Sample Approximations. 6.3.3 Comparison of Powers. 6.3.4 High-Dimensional Approximations. 6.4 Approximations Under Nonnormality. 6.4.1 Asymptotic Expansions. 6.4.2 Bootstrap Tests. 6.5 Distributions of Characteristic Roots. 6.5.1 Exact Distributions. 6.5.2 Large-Sample Case. 6.5.3 High-Dimensional Case. 6.6 Tests for Dimensionality. 6.6.1 Three Test Criteria. 6.6.2 Large-Sample and High-Dimensional Asymptotics. 6.7 High-Dimensional Tests. Problems. 7 Multivariate Regression. 7.1 Multivariate Linear Regression Model. 7.2 Statistical Inference. 7.3 Selection of Variables. 7.3.1 Stepwise Procedure. 7.3.2 C p Criterion. 7.3.3 AIC Criterion. 7.3.4 Numerical Example. 7.4 Principal Component Regression. 7.5 Selection of Response Variables. 7.6 General Linear Hypotheses and Confidence Intervals. 7.7 Penalized Regression Models. Problems. 8 Classical and High-Dimensional Tests for Covariance Matrices. 8.1 Specified Covariance Matrix. 8.1.1 Likelihood Ratio Test and Moments. 8.1.2 Asymptotic Expansions. 8.1.3 High-Dimensional Tests. 8.2 Sphericity. 8.2.1 Likelihood Ratio Tests and Moments. 8.2.2 Asymptotic Expansions. 8.2.3 High-Dimensional Tests. 8.3 Intraclass Covariance Structure. 8.3.1 Likelihood Ratio Tests and Moments. 8.3.2 Asymptotic Expansions. 8.3.3 Numerical Accuracy. 8.4 Test for Independence. 8.4.1 Likelihood Ratio Tests and Moments. 8.4.2 Asymptotic Expansions. 8.4.3 High-Dimensional Tests. 8.5 Tests for Equality of Covariance Matrices. 8.5.1 Likelihood Ratio Test and Moments. 8.5.2 Asymptotic Expansions. 8.5.3 High-Dimensional Tests. Problems. 9 Discriminant Analysis. 9.1 Classification Rules for Known Distributions. 9.2 Sample Classification Rules for Normal Populations. 9.2.1 Two Normal Populations with S 1 = S 2 . 9.2.2 Case of Several Normal Populations. 9.3 Probability of Misclassifications. 9.3.1 W-Rule. 9.3.2 Z-Rule. 9.3.3 High-Dimensional Asymptotic Results. 9.4 Canonical Discriminant Analysis. 9.4.1 Canonical Discriminant Method. 9.4.2 Test for Additional Information. 9.4.3 Selection of Variables. 9.4.4 Estimation of Dimensionality. 9.5 Regression Approach. 9.6 High-Dimensional Approach. 9.6.1 Penalized Discriminant Analysis. 9.6.2 Other Approaches. Problems. 10 Principal Component Analysis. 10.1 Definition of Principal Components. 10.2 Optimality of Principal Components. 10.3 Sample Principal Components. 10.4 MLEs of the Characteristic Roots and Vectors. 10.5 Distributions of the Characteristic Roots. 10.5.1 Exact Distribution. 10.5.2 Large-Sample Case. 10.5.3 High-Dimensional Case. 10.6 Model Selection Approach for Covariance Structures. 10.6.1 General Approach. 10.6.2 Models for Equality of the Smaller Roots. 10.6.3 Selecting a Subset of Original Variables. 10.7 Methods Related to Principal Components. 10.7.1 Fixed-Effect Principal Component Model. 10.7.2 Random-Effect Principal Components Model. Problems. 11 Canonical Correlation Analysis. 11.1 Definition of Population Canonical Correlations and Variables. 11.2 Sample Canonical Correlations. 11.3 Distributions of Canonical Correlations. 11.3.1 Distributional Reduction. 11.3.2 Large-Sample Asymptotic Distributions. 11.3.3 High-Dimensional Asymptotic Distributions. 11.3.4 Fisher's z-Transformation. 11.4 Inference for Dimensionality. 11.4.1 Test of Dimensionality. 11.4.2 Estimation of Dimensionality. 11.5 Selection of Variables. 11.5.1 Test for Redundancy. 11.5.2 Selection of Variables. Problems. 12 Growth Curve Analysis. 12.1 Growth Curve Model. 12.2 Statistical Inference: One Group. 12.2.1 Test for Adequacy. 12.2.2 Estimation and Test. 12.2.3 Confidence Intervals. 12.3 Statistical Methods: Several Groups. 12.4 Derivation of Statistical Inference. 12.4.1 A General Multivariate Linear Model. 12.4.2 Estimation. 12.4.3 LR Tests for General Linear Hypotheses. 12.4.4 Confidence Intervals. 12.5 Model Selection. 12.5.1 AIC and CAIC. 12.5.2 Derivation of CAIC. 12.5.3 Extended Growth Curve Model. Problems. 13 Approximation to the Scale-Mixted Distributions. 13.1 Introduction. 13.1.1 Simple Example: Student's t-Distribution. 13.1.2 Improving the Approximation. 13.2 Error Bounds Evaluated in Sup-Norm. 13.2.1 General Theory. 13.2.2 Scale-Mixed Normal. 13.2.3 Scale-Mixed Gamma. 13.3 Error Bounds Evaluated in L 1 -Norm. 13.3.1 Some Basic Results. 13.3.2 Scale-Mixed Normal Density. 13.3.3 Scale-Mixed Gamma Density. 13.3.4 Scale-Mixed Chi-square Density. 13.4 Multivariate Scale Mixtures. 13.4.1 General Theory. 13.4.2 Normal Case. 13.4.3 Gamma Case. Problems. 14 Approximation to Some Related Distributions. 14.1 Location and Scale Mixtures. 14.2 Maximum of Multivariate Variables. 14.2.1 Distribution of the Maximum Component of a Multivariate Variable. 14.2.2 Multivariate t-Distribution. 14.2.3 Multivariate F-Distribution. 14.3 Scale Mixtures of the F -Distribution. 14.4 Non-Uniform Error Bounds. 14.5 Method of Characteristic Functions. Problems. 15 Error Bounds for Approximations of Multivariate Tests. 15.1 Multivariate Scale Mixture and MANOVA Tests. 15.2 A Function of Multivariate Scale Mixture. 15.3 Hotelling's T? 0 Statistic. 15.4 Wilk's Lambda Distribution. 15.4.1 Univariate Case. 15.4.2 Multivariate Case. Problems. 16 Error Bounds for Approximations to Some Other Statistics. 16.1 Linear Discriminant Function. 16.1.1 Representation as Location and Scale Mixture. 16.1.2 Large-Sample Approximations. 16.1.3 High-Dimensional Approximations. 16.1.4 Some Related Topics. 16.2 Profile Analysis. 16.2.1 Parallelism Model and MLE. 16.2.2 Distributions of gamma. 16.2.3 Confidence Interval for gamma. 16.3 Estimators in the Growth Curve Model. 16.3.1 Error Bounds. 16.3.2 Distribution of the Bilinear Form. 16.4 Generalized Least Squares Estimators. Problems. Appendix. A.1 Some Results on Matrices. A.1.1 Determinants and Inverse Matrices. A.1.2 Characteristic Roots and Vectors. A.1.3 Matrix Factorizations. A.1.4 Idempotent Matrices. A.2 Inequalities and Max-Min Problems. A.3 Jacobians of Transformations. Bibliography. Index.