Fundamentals of Matrix Analysis with Applications

EDWARD BARRY SAFF, PhD, is Professor of Mathematics and Director of the Center for Constructive Approximation at Vanderbilt University. Dr. Saff is an Inaugural Fellow of the American Mathematical Society, Foreign Member of the Bulgarian Academy of Science, and the recipient of both a Guggenheim and Fulbright Fellowship. He is Editor-in-Chief of two research journals, Constructive Approximation and Computational Methods and Function Theory, and has authored or coauthored over 250 journal articles and eight books. Dr. Saff also serves as an organizer for a sequence of international research conferences that help to foster the careers of mathematicians from developing countries. ARTHUR DAVID SNIDER, PhD, PE, is Professor Emeritus at the University of South Florida, where he served on the faculties of the Departments of Mathematics, Physics, and Electrical Engineering. Previously an analyst at the Massachusetts Institute of Technology's Draper Lab and recipient of the USF Krivanek Distinguished Teacher Award, he consults in industry and has authored or coauthored over 100 journal articles and eight books. With the support of the National Science Foundation, Dr. Snider also pioneered a course in fine art appreciation for engineers.

Preface ix

Part I Introduction: Three Examples 1

1 Systems of Linear Algebraic Equations 5

1.1 Linear Algebraic Equations 5

1.2 Matrix Representation of Linear Systems and the Gauss-Jordan Algorithm 17

1.3 The Complete Gauss Elimination Algorithm 27

1.4 Echelon Form and Rank 38

1.5 Computational Considerations 46

1.6 Summary 55

2 Matrix Algebra 58

2.1 Matrix Multiplication 58

2.2 Some Physical Applications of Matrix Operators 69

2.3 The Inverse and the Transpose 76

2.4 Determinants 86

2.5 Three Important Determinant Rules 100

2.6 Summary 111

Group Projects for Part I

A. LU Factorization 116

B. Two-Point Boundary Value Problem 118

C. Electrostatic Voltage 119

D. Kirchhoff’s Laws 120

E. Global Positioning Systems 122

F. Fixed-Point Methods 123

Part II Introduction: The Structure of General Solutions to Linear Algebraic Equations 129

3 Vector Spaces 133

3.1 General Spaces Subspaces and Spans 133

3.2 Linear Dependence 142

3.3 Bases, Dimension, and Rank 151

3.4 Summary 164

4 Orthogonality 165

4.1 Orthogonal Vectors and the Gram–Schmidt Algorithm 165

4.2 Orthogonal Matrices 174

4.3 Least Squares 180

4.4 Function Spaces 190

4.5 Summary 197

Group Projects for Part II

A. Rotations and Reflections 201

B. Householder Reflectors 201

C. Infinite Dimensional Matrices 202

Part III Introduction: Reflect on This 205

5 Eigenvectors and Eigenvalues 209

5.1 Eigenvector Basics 209

5.2 Calculating Eigenvalues and Eigenvectors 217

5.3 Symmetric and Hermitian Matrices 225

5.4 Summary 232

6 Similarity 233

6.1 Similarity Transformations and Diagonalizability 233

6.2 Principle Axes and Normal Modes 244

6.3 Schur Decomposition and Its Implications 257

6.4 The Singular Value Decomposition 264

6.5 The Power Method and the QR Algorithm 282

6.6 Summary 290

7 Linear Systems of Differential Equations 293

7.1 First-Order Linear Systems 293

7.2 The Matrix Exponential Function 306

7.3 The Jordan Normal Form 316

7.4 Matrix Exponentiation via Generalized Eigenvectors 333

7.5 Summary 339

Group Projects for Part III

A. Positive Definite Matrices 342

B. Hessenberg Form 343

C. Discrete Fourier Transform 344

D. Construction of the SVD 346

E. Total Least Squares 348

F. Fibonacci Numbers 350

Answers to Odd Numbered Exercises 351

Index 393