Probability and Random Processes

VENKATARAMA KRISHNAN, PhD, is Professor Emeritus in the Department of Electrical Engineering at the University of Massachusetts Lowell. Previously, he has taught at the Indian Institute of Science, Polytechnic University, the University of Pennsylvania, Princeton University, Villanova University, and Smith College. He also worked for two years (1974-1976) as a senior systems analyst for Dynamics Research Corporation on estimation problems associated with navigation and guidance and continued as their consultant for more than a decade. Professor Krishnan's research interests include estimation of steady-state queue distributions, tomographic imaging, biosystems, and digital, aerospace, control, communications, and stochastic systems. As a senior member of IEEE, Dr. Krishnan has authored three other books in addition to technical publications.

Preface. Chapter 1: Sets, Fields, and Events. 1.1 Set Definitions. 1.2 Set Operations. 1.3 Set Algebras, Fields, and Events. Chapter 2: Probability Space and Axioms. 2.1 Probability Space. 2.2 Conditional Probability. 2.3 Independence. 2.4 Total Probability and Bayes' Theorem. Chapter 3: Basic Combinatorics. 3.1 Basic Counting Principles. 3.2 Permutations. 3.3 Combinations. Chapter 4: Discrete Distributions. 4.1 Bernoulli Trials. 4.2 Binomial Distribution. 4.3 Multinomial Distribution. 4.4 Geometric Distribution. 4.5 Negative Binomial Distribution. 4.6 Hypergeometric Distribution. 4.7 Poisson Distribution. 4.8 Logarithmic Distribution. 4.9 Summary of Discrete Distributions. Chapter 5: Random Variables. 5.1 Definition of Random Variables. 5.2 Determination of Distribution and Density Functions. 5.3 Properties of Distribution and Density Functions. 5.4 Distribution Functions from Density Functions. Chapter 6: Continuous Random Variables and Basic Distributions. 6.1 Introduction. 6.2 Uniform Distribution. 6.3 Exponential Distribution. 6.4 Normal or Gaussian Distribution. Chapter 7: Other Continuous Distributions. 7.1 Introduction. 7.2 Triangular Distribution. 7.3 Laplace Distribution. 7.4 Erlang Distribution. 7.5 Gamma Distribution. 7.6 Weibull Distribution. 7.7 Chi-Square Distribution. 7.8 Chi and Other Allied Distributions. 7.9 Student-t Density. 7.10 Snedecor F Distribution. 7.11 Lognormal Distribution. 7.12 Beta Distribution. 7.13 Cauchy Distribution. 7.14 Pareto Distribution. 7.15 Gibbs Distribution. 7.16 Mixed Distributions. 7.17 Summary of Distributions of Continuous Random Variables. Chapter 8: Conditional Densities and Distributions. 8.1 Conditional Distribution and Density for P(A) = 0. 8.2 Conditional Distribution and Density for P(A) 0. 8.3 Total Probability and Bayes' Theorem for Densities. Chapter 9: Joint Densities and Distributions. 9.1 Joint Discrete Distribution Functions. 9.2 Joint Continuous Distribution Functions 9.3 Bivariate Gaussian Distributions. Chapter 10: Moments and Conditional Moments. 10.1 Expectations. 10.2 Variance. 10.3 Means and Variances of Some Distributions. 10.4 Higher-Order Moments. 10.5 Bivariate Gaussian. Chapter 11: Characteristic Functions and Generating Functions. 11.1 Characteristic Functions. 11.2 Examples of Characteristic Functions. 11.3 Generating Functions. 11.4 Examples of Generating Functions. 11.5 Moment Generating Functions. 11.6 Cumulant Generating Functions. 11.7 Table of Means and Variances. Chapter 12: Functions of a Single Random Variable. 12.1 Random Variable g(X). 12.2 Distribution of Y = g(X ). 12.3 Direct Determination of Density fY (y) from fX(x). 12.4 Inverse Problem: Finding g(x) Given fX(x) and fY (y). 12.5 Moments of a Function of a Random Variable. Chapter 13: Functions of Multiple Random Variables. 13.1 Function of Two Random Variables, Z = g(X,Y ). 13.2 Two Functions of Two Random Variables, Z = g(X,Y ), W = h(X,Y ). 13.3 Direct Determination of Joint Density fZW(z,w ) from fXY(x,y). 13.4 Solving Z = g(X,Y ) Using an Auxiliary Random Variable. 13.5 Multiple Functions of Random Variables. Chapter 14: Inequalities, Convergences, and Limit Theorems. 14.1 Degenerate Random Variables. 14.2 Chebyshev and Allied Inequalities. 14.3 Markov Inequality. 14.4 Chernoff Bound. 14.5 Cauchy-Schwartz Inequality. 14.6 Jensen's Inequality. 14.7 Convergence Concepts. 14.8 Limit Theorems. Chapter 15: Computer Methods for Generating Random Variates. 15.1 Uniform-Distribution Random Variates. 15.2 Histograms. 15.3 Inverse Transformation Techniques. 15.4 Convolution Techniques. 15.5 Acceptance-Rejection Techniques. Chapter 16: Elements of Matrix Algebra. 16.1 Basic Theory of Matrices. 16.2 Eigenvalues and Eigenvectors of Matrices. 16.3 Vectors and Matrix Differentiations. 16.4 Block Matrices. Chapter 17: Random Vectors and Mean-Square Estimation. 17.1 Distributions and Densities. 17.2 Moments of Random Vectors. 17.3 Vector Gaussian Random Variables. 17.4 Diagonalization of Covariance Matrices. 17.5 Simultaneous Diagonalization of Covariance Matrices. 17.6 Linear Estimation of Vector Variables. Chapter 18: Estimation Theory. 18.1 Criteria of Estimators. 18.2 Estimation of Random Variables. 18.3 Estimation of Parameters (Point Estimation). 18.4 Interval Estimation (Confidence Intervals). 18.5 Hypothesis Testing (Binary). 18.6 Bayesian Estimation. Chapter 19: Random Processes. 19.1 Basic Definitions. 19.2 Stationary Random Processes. 19.3 Ergodic Processes. 19.4 Estimation of Parameters of Random Processes. 19.5 Power Spectral Density. Chapter 20: Classification of Random Processes. 20.1 Specifications of Random Processes. 20.2 Poisson Process. 20.3 Binomial Process. 20.4 Independent Increment Process. 20.5 Random-Walk Process. 20.6 Gaussian Process. 20.7 Wiener Process (Brownian Motion). 20.8 Markov Process. 20.9 Markov Chain. 20.10 Martingale Process. 20.11 Periodic Random Process. 20.12 Aperiodic Random Process (Karhunen-Loeve Expansion). Chapter 21: Random Processes and Linear Systems. 21.1 Review of Linear Systems. 21.2 Random Processes through Linear Systems. 21.3 Linear Filters. 21.4 Bandpass Stationary Random Processes. Chapter 22: Weiner and Kalman Filters. 22.1 Review of Orthogonality Principle. 22.2 Wiener Filtering. 22.3 Discrete Kalman Filter. 22.4 Continuous Kalman Filter. Chapter 23: Probabilistic Methods in Transmission Tomography. 23.1 Introduction. 23.2 Stochastic Model. 23.3 Stochastic Estimation Algorithm. 23.4 Prior Distribution P(M). 23.5 Computer Simulation. 23.6 Results and Conclusions. 23.7 Discussion of Results. 23.8 References for Chapter 23. APPENDIXES. A: A Fourier Transform Tables. B: Cumulative Gaussian Tables. C: Inverse Cumulative Gaussian Tables. D: Inverse Chi-Square Tables. E: Inverse Student-t Tables. F: Cumulative Poisson Distribution. G: Cumulative Binomial Distribution. References. Index.