Mathematical Summary for Digital Signal Processing Applications with Matlab (eBook)

Contents 8
1 Matrices 12
1.1 Properties of Vectors 13
1.2 Properties of Matrices 14
1.3 LDU Decomposition of the Matrix 18
1.4 PLDU Decomposition of an Arbitrary Matrix 21
1.5 Vector Space and Its Properties 23
1.6 Linear Independence, Span, Basis and the Dimension of the Vector Space 23
1.6.1 Linear Independence 23
1.6.2 Span 24
1.6.3 Basis 24
1.6.4 Dimension 24
1.7 Four Fundamental Vector Spaces of the Matrix 24
1.7.1 Column Space 25
1.7.2 Null Space 25
1.7.3 Row Space 25
1.7.4 Left Null Space 25
1.8 Basis of the Four Fundamental Vector Spaces of the Matrix 25
1.8.1 Column Space 26
1.9 Observations on Results of the Example 1.12 31
1.9.1 Column Space 32
1.9.2 Null Space 32
1.9.3 Left Column Space (Row Space) 32
1.9.4 Left Null Space 32
1.9.5 Observation 33
1.10 Vector Representation with Different Basis 33
1.11 Linear Transformation of the Vector 35
1.11.1 Trick to Compute the Transformation Matrix 36
1.12 Transformation Matrix with Different Basis 36
1.13 Orthogonality 37
1.13.1 Basic Definitions and Results 37
1.13.2 Orthogonal Complement 38
1.14 System of Linear Equation 38
1.15 Solutions for the System of Linear Equation [A] x=b 39
1.15.1 Trick to Obtain the Solution 40
1.16 Gram Schmidt Orthonormalization Procedure for Obtaining Orthonormal Basis 47
1.17 QR Factorization 51
1.18 Eigen Values and Eigen Vectors 53
1.19 Geometric Multiplicity (Versus) Algebraic Multiplicity 55
1.20 Diagonalization of the Matrix 58
1.21 Schur's Lemma 60
1.22 Hermitian Matrices and Skew Hermitian Matrices 61
1.23 Unitary Matrices 63
1.24 Normal Matrices 67
1.25 Applications of Diagonalization of the Non-deficient Matrix 69
1.26 Singular Value Decomposition 71
1.27 Applications of Singular Value Decomposition 73
2 Probability 78
2.1 Introduction 78
2.2 Axioms of Probability 79
2.3 Class of Events or Field (F) 79
2.4 Probability Space (S, F, P) 79
2.5 Probability Measure 79
2.6 Conditional Probability 80
2.7 Total Probability Theorem 81
2.8 Bayes Theorem 81
2.9 Independence 81
2.10 Multiple Experiments (Combined Experiments) 82
2.11 Random Variable 85
2.12 Cumulative Distribution Function (cdf) of the Random Variable `x' 86
2.13 Continuous Random Variable 87
2.14 Discrete Random Variable 87
2.15 Probability Mass Function 87
2.16 Probability Density Function 87
2.17 Two Random Variables 88
2.18 Conditional Distributions and Densities 90
2.19 Independent Random Variables 90
2.20 Some Important Results on Conditional Density Function 91
2.21 Transformation of Random Variables of the Type Y=g(X) 95
2.22 Transformation of Random Variables of the Type Y1 = g1(X1,X2), Y2 = g2(X1, X2) 96
2.23 Expectations 110
2.24 Indicator 110
2.25 Moment Generating Function 112
2.26 Characteristic Function 113
2.27 Multiple Random Variable (Random Vectors) 113
2.28 Gaussian Random Vector with Mean Vector X and Covariance Matrix CX 118
2.29 Complex Random Variables 129
2.30 Sequence of the Number and Its Convergence 130
2.31 Sequence of Functions and Its Convergence 131
2.32 Sequence of Random Variable 131
2.33 Example for the Sequence of Random Variable 133
2.34 Central Limit Theorem 133
3 Random Process 134
3.1 Introduction 134
3.2 Random Variable Xt1 135
3.3 Strictly Stationary Random Process with Order 1 135
3.4 Strictly Stationary Random Process with Order 2 135
3.5 Wide Sense Stationary Random Process 136
3.6 Complex Random Process 138
3.7 Properties of Real and Complex Random Process 138
3.8 Joint Strictly Stationary of Two Random Process 138
3.9 Jointly Wide Sense Stationary of Two Random Process 139
3.10 Correlation Matrix of the Random Column Vector XtYs for the Specific `t' `s' 139
3.11 Ergodic Process 139
3.12 Independent Random Process 143
3.13 Uncorrelated Random Process 143
3.14 Random Process as the Input and Output of the System 143
3.15 Power Spectral Density (PSD) 145
3.16 White Random Process (Noise) 148
3.17 Gaussian Random Process 149
3.18 Cyclo Stationary Random Process 150
3.19 Wide Sense Cyclo Stationary Random Process 150
3.20 Sampling and Reconstruction of Random Process 153
3.21 Band Pass Random Process 155
3.22 Random Process as the Input to the Hilbert Transformation as the System 157
3.23 Two Jointly W.S.S Low Pass Random Process Obtained Using W.S.S. Band Pass Random Process and Its Hilbert Transformation 159
4 Linear Algebra 164
4.1 Vector Space 164
4.2 Linear Transformation 165
4.3 Direct Sum 171
4.4 Transformation Matrix 173
4.5 Similar Matrices 175
4.6 Structure Theorem 177
4.7 Properties of Eigen Space 182
4.8 Properties of Generalized Eigen Space 183
4.9 Nilpotent Transformation 184
4.10 Polynomial 186
4.11 Inner Product Space 187
4.12 Orthogonal Basis 188
4.13 Riegtz Representation 190
5 Optimization 191
5.1 Constrained Optimization 191
5.2 Extension to Constrained Optimization Technique to Higher Dimensional Space with Multiple Constraints 196
5.3 Positive Definite Test of the Modified Hessian Matrix Using Eigen Value Computation 199
5.4 Constrained Optimization with Complex Numbers 203
5.5 Dual Optimization Problem 204
5.6 Kuhn-Tucker Conditions 205
6 Matlab Illustrations 207
6.1 Generation of Multivariate Gaussian Distributed Sample Outcomes with the Required Mean Vector `MY' and Covariance Matrix `CY' 207
6.2 Bacterial Foraging Optimization Technique 212
6.3 Particle Swarm Optimization 218
6.4 Newton's Iterative Method 220
6.5 Steepest Descent Algorithm 224
Index 227