Numerical Methods

Anne Greenbaum is professor of applied mathematics at the University of Washington. She is the author of Iterative Methods for Solving Linear Systems. Timothy P. Chartier is associate professor of mathematics at Davidson College.

Preface xiii Chapter 1: MATHEMATICAL MODELING 1 1.1 Modeling in Computer Animation 2 1.1.1 A Model Robe 2 1.2 Modeling in Physics: Radiation Transport 4 1.3 Modeling in Sports 6 1.4 Ecological Models 8 1.5 Modeling a Web Surfer and Google 11 1.5.1 The Vector Space Model 11 1.5.2 Google's PageRank 13 1.6 Chapter 1 Exercises 14 Chapter 2: BASIC OPERATIONS WITH MATLAB 19 2.1 Launching MATLAB 19 2.2 Vectors 20 2.3 Getting Help 22 2.4 Matrices 23 2.5 Creating and Running .m Files 24 2.6 Comments 25 2.7 Plotting 25 2.8 Creating Your Own Functions 27 2.9 Printing 28 2.10 More Loops and Conditionals 29 2.11 Clearing Variables 31 2.12 Logging Your Session 31 2.13 More Advanced Commands 31 2.14 Chapter 2 Exercises 32 Chapter 3: MONTE CARLO METHODS 41 3.1 A Mathematical Game of Cards 41 3.1.1 The Odds in Texas Holdem 42 3.2 Basic Statistics 46 3.2.1 Discrete Random Variables 48 3.2.2 Continuous Random Variables 51 3.2.3 The Central Limit Theorem 53 3.3 Monte Carlo Integration 56 3.3.1 Buffon's Needle 56 3.3.2 Estimating pi 58 3.3.3 Another Example of Monte Carlo Integration 60 3.4 Monte Carlo Simulation of Web Surfing 64 3.5 Chapter 3 Exercises 67 Chapter 4: SOLUTION OF A SINGLE NONLINEAR EQUATION IN ONE UNKNOWN 71 4.1 Bisection 75 4.2 Taylor's Theorem 80 4.3 Newton's Method 83 4.4 Quasi-Newton Methods 89 4.4.1 Avoiding Derivatives 89 4.4.2 Constant Slope Method 89 4.4.3 Secant Method 90 4.5 Analysis of Fixed Point Methods 93 4.6 Fractals, Julia Sets, and Mandelbrot Sets 98 4.7 Chapter 4 Exercises 102 Chapter 5: FLOATING-POINT ARITHMETIC 107 5.1 Costly Disasters Caused by Rounding Errors 108 5.2 Binary Representation and Base 2 Arithmetic 110 5.3 Floating-Point Representation 112 5.4 IEEE Floating-Point Arithmetic 114 5.5 Rounding 116 5.6 Correctly Rounded Floating-Point Operations 118 5.7 Exceptions 119 5.8 Chapter 5 Exercises 120 Chapter 6: CONDITIONING OF PROBLEMS; STABILITY OF ALGORITHMS 124 6.1 Conditioning of Problems 125 6.2 Stability of Algorithms 126 6.3 Chapter 6 Exercises 129 Chapter 7: DIRECT METHODS FOR SOLVING LINEAR SYSTEMS AND LEAST SQUARES PROBLEMS 131 7.1 Review of Matrix Multiplication 132 7.2 Gaussian Elimination 133 7.2.1 Operation Counts 137 7.2.2 LU Factorization 139 7.2.3 Pivoting 141 7.2.4 Banded Matrices and Matrices for Which Pivoting Is Not Required 144 7.2.5 Implementation Considerations for High Performance 148 7.3 Other Methods for Solving Ax = b 151 7.4 Conditioning of Linear Systems 154 7.4.1 Norms 154 7.4.2 Sensitivity of Solutions of Linear Systems 158 7.5 Stability of Gaussian Elimination with Partial Pivoting 164 7.6 Least Squares Problems 166 7.6.1 The Normal Equations 167 7.6.2 QR Decomposition 168 7.6.3 Fitting Polynomials to Data 171 7.7 Chapter 7 Exercises 175 Chapter 8: POLYNOMIAL AND PIECEWISE POLYNOMIAL INTERPOLATION 181 8.1 The Vandermonde System 181 8.2 The Lagrange Form of the Interpolation Polynomial 181 8.3 The Newton Form of the Interpolation Polynomial 185 8.3.1 Divided Differences 187 8.4 The Error in Polynomial Interpolation 190 8.5 Interpolation at Chebyshev Points and chebfun 192 8.6 Piecewise Polynomial Interpolation 197 8.6.1 Piecewise Cubic Hermite Interpolation 200 8.6.2 Cubic Spline Interpolation 201 8.7 Some Applications 204 8.8 Chapter 8 Exercises 206 Chapter 9: NUMERICAL DIFFERENTIATION AND RICHARDSON EXTRAPOLATION 212 9.1 Numerical Differentiation 213 9.2 Richardson Extrapolation 221 9.3 Chapter 9 Exercises 225 Chapter 10: NUMERICAL INTEGRATION 227 10.1 Newton-Cotes Formulas 227 10.2 Formulas Based on Piecewise Polynomial Interpolation 232 10.3 Gauss Quadrature 234 10.3.1 Orthogonal Polynomials 236 10.4 Clenshaw-Curtis Quadrature 240 10.5 Romberg Integration 242 10.6 Periodic Functions and the Euler-Maclaurin Formula 243 10.7 Singularities 247 10.8 Chapter 10 Exercises 248 Chapter 11: NUMERICAL SOLUTION OF THE INITIAL VALUE PROBLEM FOR ORDINARY DIFFERENTIAL EQUATIONS 251 11.1 Existence and Uniqueness of Solutions 253 11.2 One-Step Methods 257 11.2.1 Euler's Method 257 11.2.2 Higher-Order Methods Based on Taylor Series 262 11.2.3 Midpoint Method 262 11.2.4 Methods Based on Quadrature Formulas 264 11.2.5 Classical Fourth-Order Runge-Kutta and Runge-Kutta-Fehlberg Methods 265 11.2.6 An Example Using MATLAB's ODE Solver 267 11.2.7 Analysis of One-Step Methods 270 11.2.8 Practical Implementation Considerations 272 11.2.9 Systems of Equations 274 11.3 Multistep Methods 275 11.3.1 Adams-Bashforth and Adams-Moulton Methods 275 11.3.2 General Linear m-Step Methods 277 11.3.3 Linear Difference Equations 280 11.3.4 The Dahlquist Equivalence Theorem 283 11.4 Stiff Equations 284 11.4.1 Absolute Stability 285 11.4.2 Backward Differentiation Formulas (BDF Methods) 289 11.4.3 Implicit Runge-Kutta (IRK) Methods 290 11.5 Solving Systems of Nonlinear Equations in Implicit Methods 291 11.5.1 Fixed Point Iteration 292 11.5.2 Newton's Method 293 11.6 Chapter 11 Exercises 295 Chapter 12: MORE NUMERICAL LINEAR ALGEBRA: EIGENVALUES AND ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS 300 12.1 Eigenvalue Problems 300 12.1.1 The Power Method for Computing the Largest Eigenpair 310 12.1.2 Inverse Iteration 313 12.1.3 Rayleigh Quotient Iteration 315 12.1.4 The QR Algorithm 316 12.1.5 Google's PageRank 320 12.2 Iterative Methods for Solving Linear Systems 327 12.2.1 Basic Iterative Methods for Solving Linear Systems 327 12.2.2 Simple Iteration 328 12.2.3 Analysis of Convergence 332 12.2.4 The Conjugate Gradient Algorithm 336 12.2.5 Methods for Nonsymmetric Linear Systems 334 12.3 Chapter 12 Exercises 345 Chapter 13: NUMERICAL SOLUTION OF TWO-POINT BOUNDARY VALUE PROBLEMS 350 13.1 An Application: Steady-State Temperature Distribution 350 13.2 Finite Difference Methods 352 13.2.1 Accuracy 354 13.2.2 More General Equations and Boundary Conditions 360 13.3 Finite Element Methods 365 13.3.1 Accuracy 372 13.4 Spectral Methods 374 13.5 Chapter 13 Exercises 376 Chapter 14: NUMERICAL SOLUTION OF PARTIAL DIFFERENTIAL EQUATIONS 379 14.1 Elliptic Equations 381 14.1.1 Finite Difference Methods 381 14.1.2 Finite Element Methods 386 14.2 Parabolic Equations 388 14.2.1 Semidiscretization and the Method of Lines 389 14.2.2 Discretization in Time 389 14.3 Separation of Variables 396 14.3.1 Separation of Variables for Difference Equations 400 14.4 Hyperbolic Equations 402 14.4.1 Characteristics 402 14.4.2 Systems of Hyperbolic Equations 403 14.4.3 Boundary Conditions 404 14.4.4 Finite Difference Methods 404 14.5 Fast Methods for Poisson's Equation 409 14.5.1 The Fast Fourier Transform 411 14.6 Multigrid Methods 414 14.7 Chapter 14 Exercises 418 APPENDIX A REVIEW OF LINEAR ALGEBRA 421 A.1 Vectors and Vector Spaces 421 A.2 Linear Independence and Dependence 422 A.3 Span of a Set of Vectors; Bases and Coordinates; Dimension of a Vector Space 423 A.4 The Dot Product; Orthogonal and Orthonormal Sets; the Gram-Schmidt Algorithm 423 A.5 Matrices and Linear Equations 425 A.6 Existence and Uniqueness of Solutions; the Inverse; Conditions for Invertibility 427 A.7 Linear Transformations; the Matrix of a Linear Transformation 431 A.8 Similarity Transformations; Eigenvalues and Eigenvectors 432 APPENDIX B TAYLOR'S THEOREM IN MULTIDIMENSIONS 436 References 439 Index 445