Probability and Conditional Expectation (eBook)

Rolf Steyer, Institute of Psychology, University of Jena, Germany Werner Nagel, Institute of Mathematics, University of Jena, Germany

Probability and Conditional Expectation 3
Contents 9
Preface 17
Why another book on probability? 17
What is it about? 17
For whom is it? 19
Prerequisites 19
Acknowledgements 21
About the companion website 23
Part I Measure-theoretical foundations of probability theory 25
1 Measure 27
1.1 Introductory examples 27
1.2 ??????-Algebra and measurable space 28
1.2.1 ??????-Algebra generated by a set system 33
1.2.2 ??????-Algebra of Borel sets on 36
1.2.3 ??????-Algebra on a Cartesian product 38
1.2.4 ?-Stable set systems that generate a ??????-algebra 39
1.3 Measure and measure space 41
1.3.1 ??????-Additivity and related properties 42
1.3.2 Other properties 43
1.4 Specific measures 44
1.4.1 Dirac measure and counting measure 45
1.4.2 Lebesgue measure 46
1.4.3 Other examples of a measure 47
1.4.4 Finite and ??????-finite measures 48
1.4.5 Product measure 48
1.5 Continuity of a measure 49
1.6 Specifying a measure via a generating system 51
1.7 ??????-Algebra that is trivial with respect to a measure 52
1.8 Proofs 52
2 Measurable mapping 66
2.1 Image and inverse image 66
2.2 Introductory examples 67
2.2.1 Example 1: Rectangles 67
2.2.2 Example 2: Flipping two coins 69
2.3 Measurable mapping 71
2.3.1 Measurable mapping 71
2.3.2 ??????-Algebra generated by a mapping 76
2.3.3 Final ??????-algebra 79
2.3.4 Multivariate mapping 79
2.3.5 Projection mapping 81
2.3.6 Measurability with respect to a mapping 82
2.4 Theorems on measurable mappings 83
2.4.1 Measurability of a composition 84
2.4.2 Theorems on measurable functions 86
2.5 Equivalence of two mappings with respect to a measure 89
2.6 Image measure 93
2.7 Proofs 96
3 Integral 111
3.1 Definition 111
3.1.1 Integral of a nonnegative step function 111
3.1.2 Integral of a nonnegative measurable function 116
3.1.3 Integral of a measurable function 121
3.2 Properties 124
3.2.1 Integral of ??????-equivalent functions 126
3.2.2 Integral with respect to a weighted sum of measures 128
3.2.3 Integral with respect to an image measure 130
3.2.4 Convergence theorems 131
3.3 Lebesgue and Riemann integral 132
3.4 Density 134
3.5 Absolute continuity and the Radon-Nikodym theorem 136
3.6 Integral with respect to a product measure 138
3.7 Proofs 139
Part II Probability, Random Variable, and Its Distribution 155
4 Probability measure 157
4.1 Probability measure and probability space 157
4.1.1 Definition 157
4.1.2 Formal and substantive meaning of probabilistic terms 158
4.1.3 Properties of a probability measure 158
4.1.4 Examples 159
4.2 Conditional probability 162
4.2.1 Definition 162
4.2.2 Filtration and time order between events and sets of events 163
4.2.3 Multiplication rule 165
4.2.4 Examples 166
4.2.5 Theorem of total probability 167
4.2.6 Bayes’ theorem 168
4.2.7 Conditional-probability measure 169
4.3 Independence 173
4.3.1 Independence of events 173
4.3.2 Independence of set systems 174
4.4 Conditional independence given an event 175
4.4.1 Conditional independence of events given an event 176
4.4.2 Conditional independence of set systems given an event 176
4.5 Proofs 178
5 Random variable, distribution, density, and distribution function 186
5.1 Random variable and its distribution 186
5.2 Equivalence of two random variables with respect to a probability measure 192
5.2.1 Identical and P-equivalent random variables 192
5.2.2 P-equivalence, PB-equivalence, and absolute continuity 195
5.3 Multivariate random variable 198
5.4 Independence of random variables 200
5.5 Probability function of a discrete random variable 206
5.6 Probability density with respect to a measure 209
5.6.1 General concepts and properties 210
5.6.2 Density of a discrete random variable 211
5.6.3 Density of a bivariate random variable 212
5.7 Uni- or multivariate real-valued random variable 213
5.7.1 Distribution function of a univariate real-valued random variable 213
5.7.2 Distribution function of a multivariate real-valued random variable 216
5.7.3 Density of a continuous univariate real-valued random variable 217
5.7.4 Density of a continuous multivariate real-valued random variable 219
5.8 Proofs 220
6 Expectation, variance, and other moments 232
6.1 Expectation 232
6.1.1 Definition 232
6.1.2 Expectation of a discrete random variable 233
6.1.3 Computing the expectation using a density 235
6.1.4 Transformation theorem 236
6.1.5 Rules of computation 240
6.2 Moments, variance, and standard deviation 240
6.3 Proofs 245
7 Linear quasi-regression, covariance, and correlation 249
7.1 Linear quasi-regression 249
7.2 Covariance 252
7.3 Correlation 256
7.4 Expectation vector and covariance matrix 259
7.4.1 Random vector and random matrix 259
7.4.2 Expectation of a random vector and a random matrix 259
7.4.3 Covariance matrix of two multivariate random variables 261
7.5 Multiple linear quasi-regression 262
7.6 Proofs 264
8 Some distributions 278
8.1 Some distributions of discrete random variables 278
8.1.1 Discrete uniform distribution 278
8.1.2 Bernoulli distribution 279
8.1.3 Binomial distribution 280
8.1.4 Poisson distribution 282
8.1.5 Geometric distribution 284
8.2 Some distributions of continuous random variables 286
8.2.1 Continuous uniform distribution 286
8.2.2 Normal distribution 288
8.2.3 Multivariate normal distribution 291
8.2.4 Central ??????2-distribution 295
8.2.5 Central t-distribution 297
8.2.6 Central F-distribution 298
8.3 Proofs 300
Part III Conditional expectation and regression 309
9 Conditional expectation value and discrete conditional expectation 311
9.1 Conditional expectation value 311
9.2 Transformation theorem 314
9.3 Other properties 316
9.4 Discrete conditional expectation 318
9.5 Discrete regression 319
9.6 Examples 320
9.7 Proofs 325
10 Conditional expectation 329
10.1 Assumptions and definitions 329
10.2 Existence and uniqueness 331
10.2.1 Uniqueness with respect to a probability measure 332
10.2.2 A necessary and sufficient condition of uniqueness 333
10.2.3 Examples 334
10.3 Rules of computation and other properties 335
10.3.1 Rules of computation 335
10.3.2 Monotonicity 338
10.3.3 Convergence theorems 338
10.4 Factorization, regression, and conditional expectation value 340
10.4.1 Existence of a factorization 340
10.4.2 Conditional expectation and mean squared error 341
10.4.3 Uniqueness of a factorization 342
10.4.4 Conditional expectation value 343
10.5 Characterizing a conditional expectation by the joint distribution 346
10.6 Conditional mean independence 347
10.7 Proofs 352
11 Residual, conditional variance, and conditional covariance 364
11.1 Residual with respect to a conditional expectation 364
11.2 Coefficient of determination and multiple correlation 369
11.3 Conditional variance and covariance given a ??????-algebra 374
11.4 Conditional variance and covariance given a value of a random variable 375
11.5 Properties of conditional variances and covariances 378
11.6 Partial correlation 381
11.7 Proofs 383
12 Linear regression 393
12.1 Basic ideas 393
12.2 Assumptions and definitions 395
12.3 Examples 397
12.4 Linear quasi-regression 402
12.5 Uniqueness and identification of regression coefficients 404
12.6 Linear regression 405
12.7 Parameterizations of a discrete conditional expectation 407
12.8 Invariance of regression coefficients 411
12.9 Proofs 412
13 Linear logistic regression 417
13.1 Logit transformation of a conditional probability 417
13.2 Linear logistic parameterization 420
13.3 A parameterization of a discrete conditional probability 422
13.4 Identification of coefficients of a linear logistic parameterization 423
13.5 Linear logistic regression and linear logit regression 424
13.6 Proofs 431
14 Conditional expectation with respect to a conditional-probability measure 436
14.1 Introductory examples 437
14.2 Assumptions and definitions 441
14.3 Properties 447
14.4 Partial conditional expectation 448
14.5 Factorization 450
14.5.1 Conditional expectation value with respect to PB 450
14.5.2 Uniqueness of factorizations 451
14.6 Uniqueness 452
14.6.1 A necessary and sufficient condition of uniqueness 452
14.6.2 Uniqueness with respect to P and other probability measures 454
14.6.3 Necessary and sufficient conditions of P-uniqueness 454
14.6.4 Properties related to P-uniqueness 457
14.7 Conditional mean independence with respect to PZ=z 461
14.8 Proofs 463
15 Effect functions of a discrete regressor 474
15.1 Assumptions and definitions 474
15.2 Intercept function and effect functions 475
15.3 Implications of independence of X and Z for regression coefficients 478
15.4 Adjusted effect functions 480
15.5 Logit effect functions 484
15.6 Implications of independence of X and Z for the logit regression coefficients 487
15.7 Proofs 490
Part IV Conditional independence and conditional distribution 495
16 Conditional independence 497
16.1 Assumptions and definitions 497
16.1.1 Two events 498
16.1.2 Two sets of events 499
16.1.3 Two random variables 500
16.2 Properties 501
16.3 Conditional independence and conditional mean independence 509
16.4 Families of events 511
16.5 Families of set systems 512
16.6 Families of random variables 513
16.7 Proofs 517
17 Conditional distribution 529
17.1 Conditional distribution given a ??????-algebra or a random variable 529
17.2 Conditional distribution given a value of a random variable 532
17.3 Existence and uniqueness 535
17.3.1 Existence 535
17.3.2 Uniqueness of the functions PY|?????? (?, A?) 536
17.3.3 Common null set uniqueness of a conditional distribution 537
17.4 Conditional-probability measure given a value of a random variable 540
17.5 Decomposing the joint distribution of random variables 542
17.6 Conditional independence and conditional distributions 544
17.7 Expectations with respect to a conditional distribution 549
17.8 Conditional distribution function and probability density 552
17.9 Conditional distribution and Radon-Nikodym density 555
17.10 Proofs 558
References 581
List of Symbols 583
Author index 593
Subject index 594
EULA 601