Probability and Stochastic Processes

Ionut Florescu, PhD, is Research Associate Professor of Financial Engineering and Director of the Hanlon Financial Systems Lab at Stevens Institute of Technology. His areas of research interest include stochastic volatility, stochastic partial differential equations, Monte Carlo methods, and numerical methods for stochastic processes. He is also the coauthor of Handbook of Probability and coeditor of Handbook of Modeling High-Frequency Data in Finance, both published by Wiley.

List of Figures xvii

List of Tables xx

Preface xxi

Acknowledgments xxiii

Introduction 1

Part I Probability

1 Elements of Probability Measure 9

1.1 Probability Spaces 10

1.1.1 Null element of ℱ. Almost sure (a.s.) statements. Indicator of a set 21

1.2 Conditional Probability 22

1.3 Independence 29

1.4 Monotone Convergence Properties of Probability 31

1.5 Lebesgue Measure on the Unit Interval (0,1] 37

Problems 40

2 Random Variables 45

2.1 Discrete and Continuous Random Variables 48

2.2 Examples of Commonly Encountered Random Variables 52

2.3 Existence of Random Variables with Prescribed Distribution 65

2.4 Independence 68

2.5 Functions of Random Variables. Calculating Distributions 72

Problems 82

3 Applied Chapter: Generating Random Variables 87

3.1 Generating One-Dimensional Random Variables by Inverting the cdf 88

3.2 Generating One-Dimensional Normal Random Variables 91

3.3 Generating Random Variables. Rejection Sampling Method 94

3.4 Generating Random Variables. Importance Sampling 109

Problems 119

4 Integration Theory 123

4.1 Integral of Measurable Functions 124

4.2 Expectations 130

4.3 Moments of a Random Variable. Variance and the Correlation Coefficient 143

4.4 Functions of Random Variables. The Transport Formula 145

4.5 Applications. Exercises in Probability Reasoning 148

4.6 A Basic Central Limit Theorem: The DeMoivre–LaplaceTheorem: 150

Problems 152

5 Conditional Distribution and Conditional Expectation 157

5.1 Product Spaces 158

5.2 Conditional Distribution and Expectation. Calculation in Simple Cases 162

5.3 Conditional Expectation. General Definition 165

5.4 Random Vectors. Moments and Distributions 168

Problems 177

6 Moment Generating Function. Characteristic Function 181

6.1 Sums of Random Variables. Convolutions 181

6.2 Generating Functions and Applications 182

6.3 Moment Generating Function 188

6.4 Characteristic Function 192

6.5 Inversion and Continuity Theorems 199

6.6 Stable Distributions. Lvy Distribution 204

6.6.1 Truncated Lévy flight distribution 206

Problems 208

7 Limit Theorems 213

7.1 Types of Convergence 213

7.1.1 Traditional deterministic convergence types 214

7.1.2 Convergence in Lp 215

7.1.3 Almost sure (a.s.) convergence 216

7.1.4 Convergence in probability. Convergence in distribution 217

7.2 Relationships between Types of Convergence 221

7.2.1 A.S. and Lp 221

7.2.2 Probability, a.s., Lp convergence 223

7.2.3 Uniform Integrability 226

7.2.4 Weak convergence and all the others 228

7.3 Continuous Mapping Theorem. Joint Convergence. Slutsky’s Theorem 230

7.4 The Two Big Limit Theorems: LLN and CLT 232

7.4.1 A note on statistics 232

7.4.2 The order statistics 234

7.4.3 Limit theorems for the mean statistics 238

7.5 Extensions of CLT 245

7.6 Exchanging the Order of Limits and Expectations 251

Problems 252

8 Statistical Inference 259

8.1 The Classical Problems in Statistics 259

8.2 Parameter Estimation Problem 260

8.2.1 The case of the normal distribution, estimating mean when variance is unknown 262

8.2.2 The case of the normal distribution, comparing variances 264

8.3 Maximum Likelihood Estimation Method 265

8.3.1 The bisection method 267

8.4 The Method of Moments 276

8.5 Testing, the Likelihood Ratio Test 277

8.5.1 The likelihood ratio test 280

8.6 Confidence Sets 284

Problems 286

Part II Stochastic Processes

9 Introduction to Stochastic Processes 293

9.1 General Characteristics of Stochastic Processes 294

9.1.1 The index set I 294

9.1.2 The state space S 294

9.1.3 Adaptiveness, filtration, standard filtration 294

9.1.4 Pathwise realizations 296

9.1.5 The finite distribution of stochastic processes 296

9.1.6 Independent components 297

9.1.7 Stationary process 298

9.1.8 Stationary and independent increments 299

9.1.9 Other properties that characterize specific classes of stochastic processes 300

9.2 A Simple Process – The Bernoulli Process 301

Problems 304

10 The Poisson Process 307

10.1 Definitions 307

10.2 Inter-Arrival and Waiting Time for a Poisson Process 310

10.2.1 Proving that the inter-arrival times are independent 311

10.2.2 Memoryless property of the exponential distribution 315

10.2.3 Merging two independent Poisson processes 316

10.2.4 Splitting the events of the Poisson process into types 316

10.3 General Poisson Processes 317

10.3.1 Nonhomogenous Poisson process 318

10.3.2 The compound Poisson process 319

10.4 Simulation techniques. Constructing Poisson Processes 323

10.4.1 One-dimensional simple Poisson process 323

Problems 326

11 Renewal Processes 331

11.0.2 The renewal function 333

11.1 Limit Theorems for the Renewal Process 334

11.1.1 Auxiliary but very important results. Wald’s theorem. Discrete stopping time 336

11.1.2 An alternative proof of the elementary renewal theorem 340

11.2 Discrete Renewal Theory 344

11.3 The Key Renewal Theorem 349

11.4 Applications of the Renewal Theorems 350

11.5 Special cases of renewal processes 352

11.5.1 The alternating renewal process 353

11.5.2 Renewal reward process 358

11.6 The renewal Equation 359

11.7 Age-Dependent Branching processes 363

Problems 366

12 Markov Chains 371

12.1 Basic Concepts for Markov Chains 371

12.1.1 Definition 371

12.1.2 Examples of Markov chains 372

12.1.3 The Chapman– Kolmogorov equation 378

12.1.4 Communicating classes and class properties 379

12.1.5 Periodicity 379

12.1.6 Recurrence property 380

12.1.7 Types of recurrence 382

12.2 Simple Random Walk on Integers in d Dimensions 383

12.3 Limit Theorems 386

12.4 States in a MC. Stationary Distribution 387

12.4.1 Examples. Calculating stationary distribution 391

12.5 Other Issues: Graphs, First-Step Analysis 394

12.5.1 First-step analysis 394

12.5.2 Markov chains and graphs 395

12.6 A general Treatment of the Markov Chains 396

12.6.1 Time of absorption 399

12.6.2 An example 400

Problems 406

13 Semi-Markov and Continuous-time Markov Processes 411

13.1 Characterization Theorems for the General semi- Markov Process 413

13.2 Continuous-Time Markov Processes 417

13.3 The Kolmogorov Differential Equations 420

13.4 Calculating Transition Probabilities for a Markov Process General Approach 425

13.5 Limiting Probabilities for the Continuous-Time Markov Chain 426

13.6 Reversible Markov Process 429

Problems 432

14 Martingales 437

14.1 Definition and Examples 438

14.1.1 Examples of martingales 439

14.2 Martingales and Markov Chains 440

14.2.1 Martingales induced by Markov chains 440

14.3 Previsible Process. The Martingale Transform 442

14.4 Stopping Time. Stopped Process 444

14.4.1 Properties of stopping time 446

14.5 Classical Examples of Martingale Reasoning 449

14.5.1 The expected number of tosses until a binary pattern occurs 449

14.5.2 Expected number of attempts until a general pattern occurs 451

14.5.3 Gambler’s ruin probability – revisited 452

14.6 Convergence Theorems. L1 Convergence. Bounded Martingales in L2 456

Problems 458

15 Brownian Motion 465

15.1 History 465

15.2 Definition 467

15.2.1 Brownian motion as a Gaussian process 469

15.3 Properties of Brownian Motion 471

15.3.1 Hitting times. Reflection principle. Maximum value 474

15.3.2 Quadratic variation 476

15.4 Simulating Brownian Motions 480

15.4.1 Generating a Brownian motion path 480

15.4.2 Estimating parameters for a Brownian motion with drift 481

Problems 481

16 Stochastic Differential Equations 485

16.1 The Construction of the Stochastic Integral 487

16.1.1 Itȏ integral construction 490

16.1.2 An illustrative example 492

16.2 Properties of the Stochastic Integral 494

16.3 Itȏ lemma 495

16.4 Stochastic Differential Equations (SDEs) 499

16.4.1 A discussion of the types of solution for an SDE 501

16.5 Examples of SDEs 502

16.5.1 An analysis of Cox– Ingersoll– Ross (CIR) type models 507

16.5.2 Models similar to CIR 507

16.5.3 Moments calculation for the CIR model 509

16.5.4 Interpretation of the formulas for moments 511

16.5.5 Parameter estimation for the CIR model 511

16.6 Linear Systems of SDEs 513

16.7 A Simple Relationship between SDEs and Partial Differential Equations (PDEs) 515

16.8 Monte Carlo Simulations of SDEs 517

Problems 522

A Appendix: Linear Algebra and Solving Difference Equations and Systems of Differential Equations 527

A.1 Solving difference equations with constant coefficients 528

A.2 Generalized matrix inverse and pseudo-determinant 528

A.3 Connection between systems of differential equations and matrices 529

A.3.1 Writing a system of differential equations in matrix form 530

A.4 Linear Algebra results 533

A.4.1 Eigenvalues, eigenvectors of a square matrix 533

A.4.2 Matrix Exponential Function 534

A.4.3 Relationship between Exponential matrix and Eigenvectors 534

A.5 Finding fundamental solution of the homogeneous system 535

A.5.1 The case when all the eigenvalues are distinct and real 536

A.5.2 The case when some of the eigenvalues are complex 536

A.5.3 The case of repeated real eigenvalues 537

A.6 The nonhomogeneous system 538

A.6.1 The method of undetermined coefficients 538

A.6.2 The method of variation of parameters 539

A.7 Solving systems when P is non-constant 540

Bibliography 541

Index 547