Introduction to Imprecise Probabilities

Thomas Augustin, Department of Statistics, University of Munich, Germany. Frank Coolen, Department of Mathematical Sciences, Durham University, UK. Gert de Cooman, Research Professor in Uncertainty Modelling and Systems Science, Ghent University, Belgium. Matthias Troffaes, Department of Mathematical Sciences, Durham University, UK.

Introduction xiii

A brief outline of this book xv

Guide to the reader xvii

Contributors xxi

Acknowledgements xxvii

1 Desirability 1
Erik Quaeghebeur

1.1 Introduction 1

1.2 Reasoning about and with sets of desirable gambles 2

1.2.1 Rationality criteria 2

1.2.2 Assessments avoiding partial or sure loss 3

1.2.3 Coherent sets of desirable gambles 4

1.2.4 Natural extension 5

1.2.5 Desirability relative to subspaces with arbitrary vector orderings 5

1.3 Deriving and combining sets of desirable gambles 6

1.3.1 Gamble space transformations 6

1.3.2 Derived coherent sets of desirable gambles 7

1.3.3 Conditional sets of desirable gambles 8

1.3.4 Marginal sets of desirable gambles 8

1.3.5 Combining sets of desirable gambles 9

1.4 Partial preference orders 11

1.4.1 Strict preference 12

1.4.2 Nonstrict preference 12

1.4.3 Nonstrict preferences implied by strict ones 14

1.4.4 Strict preferences implied by nonstrict ones 15

1.5 Maximally committal sets of strictly desirable gambles 16

1.6 Relationships with other, nonequivalent models 18

1.6.1 Linear previsions 18

1.6.2 Credal sets 19

1.6.3 To lower and upper previsions 21

1.6.4 Simplified variants of desirability 22

1.6.5 From lower previsions 23

1.6.6 Conditional lower previsions 25

1.7 Further reading 26

Acknowledgements 27

2 Lower previsions 28
Enrique Miranda and Gert de Cooman

2.1 Introduction 28

2.2 Coherent lower previsions 29

2.2.1 Avoiding sure loss and coherence 31

2.2.2 Linear previsions 35

2.2.3 Sets of desirable gambles 39

2.2.4 Natural extension 40

2.3 Conditional lower previsions 42

2.3.1 Coherence of a finite number of conditional lower previsions 45

2.3.2 Natural extension of conditional lower previsions 47

2.3.3 Coherence of an unconditional and a conditional lower prevision 49

2.3.4 Updating with the regular extension 52

2.4 Further reading 53

2.4.1 The work of Williams 53

2.4.2 The work of Kuznetsov 54

2.4.3 The work of Weichselberger 54

Acknowledgements 55

3 Structural judgements 56
Enrique Miranda and Gert de Cooman

3.1 Introduction 56

3.2 Irrelevance and independence 57

3.2.1 Epistemic irrelevance 59

3.2.2 Epistemic independence 61

3.2.3 Envelopes of independent precise models 63

3.2.4 Strong independence 65

3.2.5 The formalist approach to independence 66

3.3 Invariance 67

3.3.1 Weak invariance 68

3.3.2 Strong invariance 69

3.4 Exchangeability 71

3.4.1 Representation theorem for finite sequences 72

3.4.2 Exchangeable natural extension 74

3.4.3 Exchangeable sequences 75

3.5 Further reading 77

3.5.1 Independence 77

3.5.2 Invariance 77

3.5.3 Exchangeability 77

Acknowledgements 78

4 Special cases 79
Sébastien Destercke and Didier Dubois

4.1 Introduction 79

4.2 Capacities and n-monotonicity 80

4.3 2-monotone capacities 81

4.4 Probability intervals on singletons 82

4.5 ∞-monotone capacities 82

4.5.1 Constructing ∞-monotone capacities 83

4.5.2 Simple support functions 83

4.5.3 Further elements 84

4.6 Possibility distributions, p-boxes, clouds and related models 84

4.6.1 Possibility distributions 84

4.6.2 Fuzzy intervals 86

4.6.3 Clouds 87

4.6.4 p-boxes 88

4.7 Neighbourhood models 89

4.7.1 Pari-mutuel 89

4.7.2 Odds-ratio 90

4.7.3 Linear-vacuous 90

4.7.4 Relations between neighbourhood models 91

4.8 Summary 91

5 Other uncertainty theories based on capacities 93
Sébastien Destercke and Didier Dubois

5.1 Imprecise probability = modal logic + probability 95

5.1.1 Boolean possibility theory and modal logic 95

5.1.2 A unifying framework for capacity based uncertainty theories 97

5.2 From imprecise probabilities to belief functions and possibility theory 97

5.2.1 Random disjunctive sets 98

5.2.2 Numerical possibility theory 100

5.2.3 Overall picture 102

5.3 Discrepancies between uncertainty theories 102

5.3.1 Objectivist vs. Subjectivist standpoints 103

5.3.2 Discrepancies in conditioning 104

5.3.3 Discrepancies in notions of independence 107

5.3.4 Discrepancies in fusion operations 109

5.4 Further reading 112

6 Game-theoretic probability 114
Vladimir Vovk and Glenn Shafer

6.1 Introduction 114

6.2 A law of large numbers 115

6.3 A general forecasting protocol 118

6.4 The axiom of continuity 122

6.5 Doob’s argument 124

6.6 Limit theorems of probability 127

6.7 Lévy’s zero-one law 128

6.8 The axiom of continuity revisited 129

6.9 Further reading 132

Acknowledgements 134

7 Statistical inference 135
Thomas Augustin, Gero Walter, and Frank P. A. Coolen

7.1 Background and introduction 136

7.1.1 What is statistical inference? 136

7.1.2 (Parametric) statistical models and i.i.d. samples 137

7.1.3 Basic tasks and procedures of statistical inference 139

7.1.4 Some methodological distinctions 140

7.1.5 Examples: Multinomial and normal distribution 141

7.2 Imprecision in statistics, some general sources and motives 143

7.2.1 Model and data imprecision; sensitivity analysis and ontological views on imprecision 143

7.2.2 The robustness shock, sensitivity analysis 144

7.2.3 Imprecision as a modelling tool to express the quality of partial knowledge 145

7.2.4 The law of decreasing credibility 145

7.2.5 Imprecise sampling models: Typical models and motives 146

7.3 Some basic concepts of statistical models relying on imprecise probabilities 147

7.3.1 Most common classes of models and notation 147

7.3.2 Imprecise parametric statistical models and corresponding i.i.d. samples 148

7.4 Generalized Bayesian inference 149

7.4.1 Some selected results from traditional Bayesian statistics 150

7.4.2 Sets of precise prior distributions, robust Bayesian inference and the generalized Bayes rule 154

7.4.3 A closer exemplary look at a popular class of models: The IDM and other models based on sets of conjugate priors in exponential families 155

7.4.4 Some further comments and a brief look at other models for generalized Bayesian inference 164

7.5 Frequentist statistics with imprecise probabilities 165

7.5.1 The nonrobustness of classical frequentist methods 166

7.5.2 (Frequentist) hypothesis testing under imprecise probability: Huber-Strassen theory and extensions 169

7.5.3 Towards a frequentist estimation theory under imprecise probabilities – some basic criteria and first results 171

7.5.4 A brief outlook on frequentist methods 174

7.6 Nonparametric predictive inference 175

7.6.1 Overview 175

7.6.2 Applications and challenges 177

7.7 A brief sketch of some further approaches and aspects 178

7.8 Data imprecision, partial identification 179

7.8.1 Data imprecision 180

7.8.2 Cautious data completion 181

7.8.3 Partial identification and observationally equivalent models 183

7.8.4 A brief outlook on some further aspects 186

7.9 Some general further reading 187

7.10 Some general challenges 188

Acknowledgements 189

8 Decision making 190
Nathan Huntley, Robert Hable, and Matthias C. M. Troffaes

8.1 Non-sequential decision problems 190

8.1.1 Choosing from a set of gambles 191

8.1.2 Choice functions for coherent lower previsions 192

8.2 Sequential decision problems 197

8.2.1 Static sequential solutions: Normal form 198

8.2.2 Dynamic sequential solutions: Extensive form 199

8.3 Examples and applications 202

8.3.1 Ellsberg’s paradox 202

8.3.2 Robust Bayesian statistics 205

9 Probabilistic graphical models 207
Alessandro Antonucci, Cassio P. de Campos, and Marco Zaffalon

9.1 Introduction 207

9.2 Credal sets 208

9.2.1 Definition and relation with lower previsions 208

9.2.2 Marginalization and conditioning 210

9.2.3 Composition 212

9.3 Independence 213

9.4 Credal networks 215

9.4.1 Nonseparately specified credal networks 217

9.5 Computing with credal networks 220

9.5.1 Credal networks updating 220

9.5.2 Modelling and updating with missing data 221

9.5.3 Algorithms for credal networks updating 223

9.5.4 Inference on credal networks as a multilinear programming task 224

9.6 Further reading 226

Acknowledgements 229

10 Classification 230
Giorgio Corani, Joaquín Abellán, Andrés Masegosa, Serafin Moral, and Marco Zaffalon

10.1 Introduction 230

10.2 Naive Bayes 231

10.2.1 Derivation of naive Bayes 232

10.3 Naive credal classifier (NCC) 233

10.3.1 Checking Credal-dominance 233

10.3.2 Particular behaviours of NCC 235

10.3.3 NCC2: Conservative treatment of missing data 236

10.4 Extensions and developments of the naive credal classifier 237

10.4.1 Lazy naive credal classifier 237

10.4.2 Credal model averaging 238

10.4.3 Profile-likelihood classifiers 239

10.4.4 Tree-augmented networks (TAN) 240

10.5 Tree-based credal classifiers 242

10.5.1 Uncertainty measures on credal sets: The maximum entropy function 242

10.5.2 Obtaining conditional probability intervals with the imprecise Dirichlet model 245

10.5.3 Classification procedure 246

10.6 Metrics, experiments and software 249

10.7 Scoring the conditional probability of the class 251

10.7.1 Software 251

10.7.2 Experiments 251

10.7.3 Experiments comparing conditional probabilities of the class 253

Acknowledgements 257

11 Stochastic processes 258
Filip Hermans and Damjan Škulj

11.1 The classical characterization of stochastic processes 258

11.1.1 Basic definitions 258

11.1.2 Precise Markov chains 259

11.2 Event-driven random processes 261

11.3 Imprecise Markov chains 263

11.3.1 From precise to imprecise Markov chains 264

11.3.2 Imprecise Markov models under epistemic irrelevance 265

11.3.3 Imprecise Markov models under strong independence 268

11.3.4 When does the interpretation of independence (not) matter? 270

11.4 Limit behaviour of imprecise Markov chains 272

11.4.1 Metric properties of imprecise probability models 272

11.4.2 The Perron-Frobenius theorem 273

11.4.3 Invariant distributions 274

11.4.4 Coefficients of ergodicity 275

11.4.5 Coefficients of ergodicity for imprecise Markov chains 275

11.5 Further reading 277

12 Financial risk measurement 279
Paolo Vicig

12.1 Introduction 279

12.2 Imprecise previsions and betting 280

12.3 Imprecise previsions and risk measurement 282

12.3.1 Risk measures as imprecise previsions 283

12.3.2 Coherent risk measures 284

12.3.3 Convex risk measures (and previsions) 285

12.4 Further reading 289

13 Engineering 291
Michael Oberguggenberger

13.1 Introduction 291

13.2 Probabilistic dimensioning in a simple example 295

13.3 Random set modelling of the output variability 298

13.4 Sensitivity analysis 300

13.5 Hybrid models 301

13.6 Reliability analysis and decision making in engineering 302

13.7 Further reading 303

14 Reliability and risk 305
Frank P. A. Coolen and Lev V. Utkin

14.1 Introduction 305

14.2 Stress-strength reliability 306

14.3 Statistical inference in reliability and risk 310

14.4 Nonparametric predictive inference in reliability and risk 312

14.5 Discussion and research challenges 317

15 Elicitation 318
Michael Smithson

15.1 Methods and issues 318

15.2 Evaluating imprecise probability judgements 322

15.3 Factors affecting elicitation 324

15.4 Matching methods with purposes 327

15.5 Further reading 328

16 Computation 329
Matthias C. M. Troffaes and Robert Hable

16.1 Introduction 329

16.2 Natural extension 329

16.2.1 Conditional lower previsions with arbitrary domains 330

16.2.2 The Walley–Pelessoni–Vicig algorithm 331

16.2.3 Choquet integration 332

16.2.4 Möbius inverse 334

16.2.5 Linear-vacuous mixture 334

16.3 Decision making 335

16.3.1 Γ-maximin, Γ-maximax and Hurwicz 335

16.3.2 Maximality 335

16.3.3 E-admissibility 336

16.3.4 Interval dominance 337

References 338

Author index 375

Subject index 385