Discrete Stochastic Processes and Optimal Filtering

Jean-Claude Bertein has a Master's and PhD degree from the University of Paris VI. He was formerly a research engineer at Alcatel and today is Professor in the Department of Mathematics and Physics at the Graduate School of Electrical and Electronic Engineering (ESIEE) Paris. Roger Ceschi is an ENSEA engineer and holds a Master's and PhD degree from the University of Paris XI. He was formerly Director of the ENSEA and today he is Director General of the ESIEE Amiens. He is also the author of a theorem on analytic signals and is Visiting Professor at the BIPT and BIT Universities in China.

Preface ix

Introduction xi

Chapter 1. Random Vectors 1

1.1. Definitions and general properties. 1

1.2. Spaces L1 (dP) and L2 (dP) 20

1.3. Mathematical expectation and applications 23

1.4. Second order random variables and vectors. 39

1.5. Linear independence of vectors of L2 (dP) 46

1.6. Conditional expectation (concerning random vectors with density function) 51

1.7. Exercises for Chapter 1 56

Chapter 2. Gaussian Vectors 63

2.1. Some reminders regarding random Gaussian vectors 63

2.2. Definition and characterization of Gaussian vectors 66

2.3. Results relative to independence 68

2.4. Affine transformation of a Gaussian vector 72

2.5. The existence of Gaussian vectors. 74

2.6. Exercises for Chapter 2 84

Chapter 3. Introduction to Discrete Time Processes 93

3.1. Definition 93

3.2. WSS processes and spectral measure 105

3.3. Spectral representation of a WSS process 109

3.4. Introduction to digital filtering 114

3.5. Important example: autoregressive process 127

3.6. Exercises for Chapter 3 132

Chapter 4. Estimation 139

4.1. Position of the problem 139

4.2. Linear estimation 142

4.3. Best estimate – conditional expectation 154

4.4. Example: prediction of an autoregressive process AR (1) 162

4.5. Multivariate processes 163

4.6. Exercises for Chapter 4 172

Chapter 5. The Wiener Filter 177

5.1. Introduction 177

5.2. Resolution and calculation of the FIR filter 179

5.3. Evaluation of the least error 181

5.4. Resolution and calculation of the IIR filter 183

5.5. Evaluation of least mean square error 187

5.6. Exercises for Chapter 5 188

Chapter 6. Adaptive Filtering: Algorithm of the Gradient and the LMS 195

6.1. Introduction 195

6.2. Position of problem 198

6.3. Data representation 200

6.4. Minimization of the cost function 202

6.5. Gradient algorithm 209

6.6. Geometric interpretation 212

6.7. Stability and convergence 216

6.8. Estimation of gradient and LMS algorithm 221

6.9. Example of the application of the LMS algorithm 224

6.10. Exercises for Chapter 6 233

Chapter 7. The Kalman Filter 235

7.1. Position of problem 235

7.2. Approach to estimation 239

7.3. Kalman filtering 243

7.4. Exercises for Chapter 7 261

7.5. Appendices 267

7.6. Examples treated using Matlab software 273

Table of Symbols and Notations 281

Bibliography 283

Index 285