Handbook of Probability

IONUT FLORESCU, PhD, is Research Associate Professor of Financial Engineering and Director of the Hanlon Financial Systems Lab at Stevens Institute of Technology. He has published extensively in his areas of research interest, which include stochastic volatility, stochastic partial differential equations, Monte Carlo methods, and numerical methods for stochastic processes. CIPRIAN A. TUDOR, PhD, is Professor of Mathematics at the University of Lille 1, France. His research interests include Brownian motion, limit theorems, statistical inference for stochastic processes, and financial mathematics. He has over eighty scientific publications in various internationally recognized journals on probability theory and statistics. He serves as a referee for over a dozen journals and has spoken at more than thirty-five conferences worldwide.

List of Figures xv

Preface xvii

Introduction xix

1 Probability Space 1

1.1 Introduction/Purpose of the Chapter 1

1.2 Vignette/Historical Notes 2

1.3 Notations and Definitions 2

1.4 Theory and Applications 4

1.4.1 Algebras 4

1.4.2 Sigma Algebras 5

1.4.3 Measurable Spaces 7

1.4.4 Examples 7

1.4.5 The Borel _-Algebra 9

1.5 Summary 12

Exercises 12

2 Probability Measure 15

2.1 Introduction/Purpose of the Chapter 15

2.2 Vignette/Historical Notes 16

2.3 Theory and Applications 17

2.3.1 Definition and Basic Properties 17

2.3.2 Uniqueness of Probability Measures 22

2.3.3 Monotone Class 24

2.3.4 Examples 26

2.3.5 Monotone Convergence Properties of Probability 28

2.3.6 Conditional Probability 31

2.3.7 Independence of Events and _-Fields 39

2.3.8 Borel–Cantelli Lemmas 46

2.3.9 Fatou’s Lemmas 48

2.3.10 Kolmogorov’s Zero–One Law 49

2.4 Lebesgue Measure on the Unit Interval (01] 50

Exercises 52

3 Random Variables: Generalities 63

3.1 Introduction/Purpose of the Chapter 63

3.2 Vignette/Historical Notes 63

3.3 Theory and Applications 64

3.3.1 Definition 64

3.3.2 The Distribution of a Random Variable 65

3.3.3 The Cumulative Distribution Function of a Random Variable 67

3.3.4 Independence of Random Variables 70

Exercises 71

4 Random Variables: The Discrete Case 79

4.1 Introduction/Purpose of the Chapter 79

4.2 Vignette/Historical Notes 80

4.3 Theory and Applications 80

4.3.1 Definition and Basic Facts 80

4.3.2 Moments 84

4.4 Examples of Discrete Random Variables 89

4.4.1 The (Discrete) Uniform Distribution 89

4.4.2 Bernoulli Distribution 91

4.4.3 Binomial (n p) Distribution 92

4.4.4 Geometric (p) Distribution 95

4.4.5 Negative Binomial (r p) Distribution 101

4.4.6 Hypergeometric Distribution (N m n) 102

4.4.7 Poisson Distribution 104

Exercises 108

5 Random Variables: The Continuous Case 119

5.1 Introduction/Purpose of the Chapter 119

5.2 Vignette/Historical Notes 119

5.3 Theory and Applications 120

5.3.1 Probability Density Function (p.d.f.) 120

5.3.2 Cumulative Distribution Function (c.d.f.) 124

5.3.3 Moments 127

5.3.4 Distribution of a Function of the Random Variable 128

5.4 Examples 130

5.4.1 Uniform Distribution on an Interval [ab] 130

5.4.2 Exponential Distribution 133

5.4.3 Normal Distribution (_ _2) 136

5.4.4 Gamma Distribution 139

5.4.5 Beta Distribution 144

5.4.6 Student’s t Distribution 147

5.4.7 Pareto Distribution 149

5.4.8 The Log-Normal Distribution 151

5.4.9 Laplace Distribution 153

5.4.10 Double Exponential Distribution 155

Exercises 156

6 Generating Random Variables 177

6.1 Introduction/Purpose of the Chapter 177

6.2 Vignette/Historical Notes 178

6.3 Theory and Applications 178

6.3.1 Generating One-Dimensional Random Variables by Inverting the Cumulative Distribution Function (c.d.f.) 178

6.3.2 Generating One-Dimensional Normal Random Variables 183

6.3.3 Generating Random Variables. Rejection Sampling Method 186

6.3.4 Generating from a Mixture of Distributions 193

6.3.5 Generating Random Variables. Importance Sampling 195

6.3.6 Applying Importance Sampling 198

6.3.7 Practical Consideration: Normalizing Distributions 201

6.3.8 Sampling Importance Resampling 203

6.3.9 Adaptive Importance Sampling 204

6.4 Generating Multivariate Distributions with Prescribed Covariance Structure 205

Exercises 208

7 Random Vectors in Rn 210

7.1 Introduction/Purpose of the Chapter 210

7.2 Vignette/Historical Notes 210

7.3 Theory and Applications 211

7.3.1 The Basics 211

7.3.2 Marginal Distributions 212

7.3.3 Discrete Random Vectors 214

7.3.4 Multinomial Distribution 219

7.3.5 Testing Whether Counts are Coming from a Specific Multinomial Distribution 220

7.3.6 Independence 221

7.3.7 Continuous Random Vectors 223

7.3.8 Change of Variables. Obtaining Densities of Functions of Random Vectors 229

7.3.9 Distribution of Sums of Random Variables. Convolutions 231

Exercises 236

8 Characteristic Function 255

8.1 Introduction/Purpose of the Chapter 255

8.2 Vignette/Historical Notes 255

8.3 Theory and Applications 256

8.3.1 Definition and Basic Properties 256

8.3.2 The Relationship Between the Characteristic Function and the Distribution 260

8.4 Calculation of the Characteristic Function for Commonly Encountered Distributions 265

8.4.1 Bernoulli and Binomial 265

8.4.2 Uniform Distribution 266

8.4.3 Normal Distribution 267

8.4.4 Poisson Distribution 267

8.4.5 Gamma Distribution 268

8.4.6 Cauchy Distribution 269

8.4.7 Laplace Distribution 270

8.4.8 Stable Distributions. L´evy Distribution 271

8.4.9 Truncated L´evy Flight Distribution 274

Exercises 275

9 Moment-Generating Function 280

9.1 Introduction/Purpose of the Chapter 280

9.2 Vignette/Historical Notes 280

9.3 Theory and Applications 281

9.3.1 Generating Functions and Applications 281

9.3.2 Moment-Generating Functions. Relation with the Characteristic Functions 288

9.3.3 Relationship with the Characteristic Function 292

9.3.4 Properties of the MGF 292

Exercises 294

10 Gaussian Random Vectors 300

10.1 Introduction/Purpose of the Chapter 300

10.2 Vignette/Historical Notes 301

10.3 Theory and Applications 301

10.3.1 The Basics 301

10.3.2 Equivalent Definitions of a Gaussian Vector 303

10.3.3 Uncorrelated Components and Independence 309

10.3.4 The Density of a Gaussian Vector 313

10.3.5 Cochran’s Theorem 316

10.3.6 Matrix Diagonalization and Gaussian Vectors 319

Exercises 325

11 Convergence Types. Almost Sure Convergence. Lp-Convergence. Convergence in Probability 338

11.1 Introduction/Purpose of the Chapter 338

11.2 Vignette/Historical Notes 339

11.3 Theory and Applications: Types of Convergence 339

11.3.1 Traditional Deterministic Convergence Types 339

11.3.2 Convergence of Moments of an r.v.—Convergence in Lp 341

11.3.3 Almost Sure (a.s.) Convergence 342

11.3.4 Convergence in Probability 344

11.4 Relationships Between Types of Convergence 346

11.4.1 a.s. and Lp 347

11.4.2 Probability and a.s./Lp 351

11.4.3 Uniform Integrability 357

Exercises 359

12 Limit Theorems 372

12.1 Introduction/Purpose of the Chapter 372

12.2 Vignette/Historical Notes 372

12.3 Theory and Applications 375

12.3.1 Weak Convergence 375

12.3.2 The Law of Large Numbers 384

12.4 Central Limit Theorem 401

Exercises 409

13 Appendix A: Integration Theory. General Expectations 421

13.1 Integral of Measurable Functions 422

13.1.1 Integral of Simple (Elementary) Functions 422

13.1.2 Integral of Positive Measurable Functions 424

13.1.3 Integral of Measurable Functions 428

13.2 General Expectations and Moments of a Random Variable 429

13.2.1 Moments and Central Moments. Lp Space 430

13.2.2 Variance and the Correlation Coefficient 431

13.2.3 Convergence Theorems 433

14 Appendix B: Inequalities Involving Random Variables and Their Expectations 434

14.1 Functions of Random Variables. The Transport Formula 441

Bibliography 445

Index 447