Case Studies in Bayesian Statistical Modelling and Analysis

Clair Alston, Queensland University of Technology and Science, Australia. Kerrie L. Mengersen, Queensland University of Technology and Science, Australia. Tony Pettitt, Queensland University of Technology and Science, Australia.

Preface xvii

List of contributors xix

1 Introduction 1
Clair L. Alston, Margaret Donald, Kerrie L. Mengersen and Anthony N. Pettitt

1.1 Introduction 1

1.2 Overview 1

1.3 Further reading 8

1.3.1 Bayesian theory and methodology 8

1.3.2 Bayesian methodology 10

1.3.3 Bayesian computation 10

1.3.4 Bayesian software 11

1.3.5 Applications 13

References 13

2 Introduction to MCMC 17
Anthony N. Pettitt and Candice M. Hincksman

2.1 Introduction 17

2.2 Gibbs sampling 18

2.2.1 Example: Bivariate normal 18

2.2.2 Example: Change-point model 19

2.3 Metropolis–Hastings algorithms 19

2.3.1 Example: Component-wise MH or MH within Gibbs 20

2.3.2 Extensions to basic MCMC 21

2.3.3 Adaptive MCMC 22

2.3.4 Doubly intractable problems 22

2.4 Approximate Bayesian computation 24

2.5 Reversible jump MCMC 25

2.6 MCMC for some further applications 26

References 27

3 Priors: Silent or active partners of Bayesian inference? 30
Samantha Low Choy

3.1 Priors in the very beginning 30

3.1.1 Priors as a basis for learning 32

3.1.2 Priors and philosophy 32

3.1.3 Prior chronology 33

3.1.4 Pooling prior information 34

3.2 Methodology I: Priors defined by mathematical criteria 35

3.2.1 Conjugate priors 35

3.2.2 Impropriety and hierarchical priors 37

3.2.3 Zellner’s g-prior for regression models 37

3.2.4 Objective priors 38

3.3 Methodology II: Modelling informative priors 40

3.3.1 Informative modelling approaches 40

3.3.2 Elicitation of distributions 42

3.4 Case studies 44

3.4.1 Normal likelihood: Time to submit research dissertations 44

3.4.2 Binomial likelihood: Surveillance for exotic plant pests 47

3.4.3 Mixture model likelihood: Bioregionalization 50

3.4.4 Logistic regression likelihood: Mapping species distribution via habitat models 53

3.5 Discussion 57

3.5.1 Limitations 57

3.5.2 Finding out about the problem 58

3.5.3 Prior formulation 59

3.5.4 Communication 60

3.5.5 Conclusion 61

Acknowledgements 61

References 61

4 Bayesian analysis of the normal linear regression model 66
Christopher M. Strickland and Clair L. Alston

4.1 Introduction 66

4.2 Case studies 67

4.2.1 Case study 1: Boston housing data set 67

4.2.2 Case study 2: Production of cars and station wagons 67

4.3 Matrix notation and the likelihood 67

4.4 Posterior inference 68

4.4.1 Natural conjugate prior 69

4.4.2 Alternative prior specifications 73

4.4.3 Generalizations of the normal linear model 74

4.4.4 Variable selection 78

4.5 Analysis 81

4.5.1 Case study 1: Boston housing data set 81

4.5.2 Case study 2: Car production data set 85

References 88

5 Adapting ICU mortality models for local data: A Bayesian approach 90
Petra L. Graham, Kerrie L. Mengersen and David A. Cook

5.1 Introduction 90

5.2 Case study: Updating a known risk-adjustment model for local use 91

5.3 Models and methods 92

5.4 Data analysis and results 96

5.4.1 Updating using the training data 96

5.4.2 Updating the model yearly 98

5.5 Discussion 100

References 101

6 A Bayesian regression model with variable selection for genome-wide association studies 103
Carla Chen, Kerrie L. Mengersen, Katja Ickstadt and Jonathan M. Keith

6.1 Introduction 103

6.2 Case study: Case–control of Type 1 diabetes 104

6.3 Case study: GENICA 105

6.4 Models and methods 105

6.4.1 Main effect models 105

6.4.2 Main effects and interactions 108

6.5 Data analysis and results 109

6.5.1 WTCCC TID 109

6.5.2 GENICA 110

6.6 Discussion 112

Acknowledgements 115

References 115

6.A  Appendix: SNP IDs 117

7 Bayesian meta-analysis 118
Jegar O. Pitchforth and Kerrie L. Mengersen

7.1 Introduction 118

7.2 Case study 1: Association between red meat consumption and breast cancer 119

7.2.1 Background 119

7.2.2 Meta-analysis models 121

7.2.3 Computation 125

7.2.4 Results 125

7.2.5 Discussion 129

7.3 Case study 2: Trends in fish growth rate and size 130

7.3.1 Background 130

7.3.2 Meta-analysis models 131

7.3.3 Computation 134

7.3.4 Results 134

7.3.5 Discussion 135

Acknowledgements 137

References 138

8 Bayesian mixed effects models 141
Clair L. Alston, Christopher M. Strickland, Kerrie L. Mengersen and Graham E. Gardner

8.1 Introduction 141

8.2 Case studies 142

8.2.1 Case study 1: Hot carcase weight of sheep carcases 142

8.2.2 Case study 2: Growth of primary school girls 142

8.3 Models and methods 146

8.3.1 Model for Case study 1 147

8.3.2 Model for Case study 2 148

8.3.3 MCMC estimation 149

8.4 Data analysis and results 150

8.5 Discussion 158

References 158

9 Ordering of hierarchies in hierarchical models: Bone mineral density estimation 159
Cathal D. Walsh and Kerrie L. Mengersen

9.1 Introduction 159

9.2 Case study 160

9.2.1 Measurement of bone mineral density 160

9.3 Models 161

9.3.1 Hierarchical model 162

9.3.2 Model H1 163

9.3.3 Model H2 163

9.4 Data analysis and results 164

9.4.1 Model H1 164

9.4.2 Model H2 165

9.4.3 Implication of ordering 166

9.4.4 Simulation study 166

9.4.5 Study design 166

9.4.6 Simulation study results 167

9.5 Discussion 168

References 168

9.A  Appendix: Likelihoods 170

10 Bayesian Weibull survival model for gene expression data 171
Sri Astuti Thamrin, James M. McGree and Kerrie L. Mengersen

10.1 Introduction 171

10.2 Survival analysis 172

10.3 Bayesian inference for the Weibull survival model 174

10.3.1 Weibull model without covariates 174

10.3.2 Weibull model with covariates 175

10.3.3 Model evaluation and comparison 176

10.4 Case study 178

10.4.1 Weibull model without covariates 178

10.4.2 Weibull survival model with covariates 180

10.4.3 Model evaluation and comparison 182

10.5 Discussion 182

References 183

11 Bayesian change point detection in monitoring clinical outcomes 186
Hassan Assareh, Ian Smith and Kerrie L. Mengersen

11.1 Introduction 186

11.2 Case study: Monitoring intensive care unit outcomes 187

11.3 Risk-adjusted control charts 187

11.4 Change point model 188

11.5 Evaluation 189

11.6 Performance analysis 190

11.7 Comparison of Bayesian estimator with other methods 194

11.8 Conclusion 194

References 195

12 Bayesian splines 197
Samuel Clifford and Samantha Low Choy

12.1 Introduction 197

12.2 Models and methods 197

12.2.1 Splines and linear models 197

12.2.2 Link functions 198

12.2.3 Bayesian splines 198

12.2.4 Markov chain Monte Carlo 204

12.2.5 Model choice 206

12.2.6 Posterior diagnostics 207

12.3 Case studies 207

12.3.1 Data 207

12.3.2 Analysis 208

12.4 Conclusion 216

12.4.1 Discussion 216

12.4.2 Extensions 217

12.4.3 Summary 218

References 218

13 Disease mapping using Bayesian hierarchical models 221
Arul Earnest, Susanna M. Cramb and Nicole M. White

13.1 Introduction 221

13.2 Case studies 224

13.2.1 Case study 1: Spatio-temporal model examining the incidence of birth defects 224

13.2.2 Case study 2: Relative survival model examining survival from breast cancer 225

13.3 Models and methods 225

13.3.1 Case study 1 225

13.3.2 Case study 2 229

13.4 Data analysis and results 230

13.4.1 Case study 1 230

13.4.2 Case study 2 231

13.5 Discussion 234

References 237

14 Moisture, crops and salination: An analysis of a three-dimensional agricultural data set 240
Margaret Donald, Clair L. Alston, Rick Young and Kerrie L. Mengersen

14.1 Introduction 240

14.2 Case study 241

14.2.1 Data 242

14.2.2 Aim of the analysis 242

14.3 Review 243

14.3.1 General methodology 243

14.3.2 Computations 243

14.4 Case study modelling 243

14.4.1 Modelling framework 243

14.5 Model implementation: Coding considerations 246

14.5.1 Neighbourhood matrices and CAR models 246

14.5.2 Design matrices vs indexing 246

14.6 Case study results 247

14.7 Conclusions 249

References 250

15 A Bayesian approach to multivariate state space modelling: A study of a Fama–French asset-pricing model with time-varying regressors 252
Christopher M. Strickland and Philip Gharghori

15.1 Introduction 252

15.2 Case study: Asset pricing in financial markets 253

15.2.1 Data 254

15.3 Time-varying Fama–French model 254

15.3.1 Specific models under consideration 255

15.4 Bayesian estimation 256

15.4.1 Gibbs sampler 256

15.4.2 Sampling Σε 257

15.4.3 Sampling β 1:n 257

15.4.4 Sampling Σ  α 259

15.4.5 Likelihood calculation 260

15.5 Analysis 261

15.5.1 Prior elicitation 261

15.5.2 Estimation output 261

15.6 Conclusion 264

References 265

16 Bayesian mixture models: When the thing you need to know is the thing you cannot measure 267
Clair L. Alston, Kerrie L. Mengersen and Graham E. Gardner

16.1 Introduction 267

16.2 Case study: CT scan images of sheep 268

16.3 Models and methods 270

16.3.1 Bayesian mixture models 270

16.3.2 Parameter estimation using the Gibbs sampler 273

16.3.3 Extending the model to incorporate spatial information 274

16.4 Data analysis and results 276

16.4.1 Normal Bayesian mixture model 276

16.4.2 Spatial mixture model 278

16.4.3 Carcase volume calculation 281

16.5 Discussion 284

References 284

17 Latent class models in medicine 287
Margaret Rolfe, Nicole M. White and Carla Chen

17.1 Introduction 287

17.2 Case studies 288

17.2.1 Case study 1: Parkinson’s disease 288

17.2.2 Case study 2: Cognition in breast cancer 288

17.3 Models and methods 289

17.3.1 Finite mixture models 290

17.3.2 Trajectory mixture models 292

17.3.3 Goodness of fit 296

17.3.4 Label switching 297

17.3.5 Model computation 298

17.4 Data analysis and results 300

17.4.1 Case study 1: Phenotype identification in PD 300

17.4.2 Case study 2: Trajectory groups for verbal memory 302

17.5 Discussion 306

References 307

18 Hidden Markov models for complex stochastic processes: A case study in electrophysiology 310
Nicole M. White, Helen Johnson, Peter Silburn, Judith Rousseau and Kerrie L. Mengersen

18.1 Introduction 310

18.2 Case study: Spike identification and sorting of extracellular recordings 311

18.3 Models and methods 312

18.3.1 What is an HMM? 312

18.3.2 Modelling a single AP: Application of a simple HMM 313

18.3.3 Multiple neurons: An application of a factorial HMM 315

18.3.4 Model estimation and inference 317

18.4 Data analysis and results 320

18.4.1 Simulation study 320

18.4.2 Case study: Extracellular recordings collected during deep brain stimulation 323

18.5 Discussion 326

References 327

19 Bayesian classification and regression trees 330
Rebecca A. O’Leary, Samantha Low Choy, Wenbiao Hu and Kerrie L. Mengersen

19.1 Introduction 330

19.2 Case studies 332

19.2.1 Case study 1: Kyphosis 332

19.2.2 Case study 2: Cryptosporidium 332

19.3 Models and methods 334

19.3.1 CARTs 334

19.3.2 Bayesian CARTs 335

19.4 Computation 337

19.4.1 Building the BCART model – stochastic search 337

19.4.2 Model diagnostics and identifying good trees 339

19.5 Case studies – results 341

19.5.1 Case study 1: Kyphosis 341

19.5.2 Case study 2: Cryptosporidium 343

19.6 Discussion 345

References 346

20 Tangled webs: Using Bayesian networks in the fight against infection 348
Mary Waterhouse and Sandra Johnson

20.1 Introduction to Bayesian network modelling 348

20.1.1 Building a BN 349

20.2 Introduction to case study 351

20.3 Model 352

20.4 Methods 354

20.5 Results 355

20.6 Discussion 357

References 359

21 Implementing adaptive dose finding studies using sequential Monte Carlo 361
James M. McGree, Christopher C. Drovandi and Anthony N. Pettitt

21.1 Introduction 361

21.2 Model and priors 363

21.3 SMC for dose finding studies 364

21.3.1 Importance sampling 364

21.3.2 SMC 365

21.3.3 Dose selection procedure 367

21.4 Example 369

21.5 Discussion 371

References 372

21.A Appendix: Extra example 373

22 Likelihood-free inference for transmission rates of nosocomial pathogens 374
Christopher C. Drovandi and Anthony N. Pettitt

22.1 Introduction 374

22.2 Case study: Estimating transmission rates of nosocomial pathogens 375

22.2.1 Background 375

22.2.2 Data 376

22.2.3 Objective 376

22.3 Models and methods 376

22.3.1 Models 376

22.3.2 Computing the likelihood 379

22.3.3 Model simulation 380

22.3.4 ABC 381

22.3.5 ABC algorithms 382

22.4 Data analysis and results 384

22.5 Discussion 385

References 386

23 Variational Bayesian inference for mixture models 388
Clare A. McGrory

23.1 Introduction 388

23.2 Case study: Computed tomography (CT) scanning of a loin portion of a pork carcase 390

23.3 Models and methods 392

23.4 Data analysis and results 397

23.5 Discussion 399

References 399

23.A Appendix: Form of the variational posterior for a mixture of multivariate normal densities 401

24 Issues in designing hybrid algorithms 403
Jeong E. Lee, Kerrie L. Mengersen and Christian P. Robert

24.1 Introduction 403

24.2 Algorithms and hybrid approaches 406

24.2.1 Particle system in the MCMC context 407

24.2.2 MALA 407

24.2.3 DRA 408

24.2.4 PS 409

24.2.5 Population Monte Carlo (PMC) algorithm 410

24.3 Illustration of hybrid algorithms 412

24.3.1 Simulated data set 412

24.3.2 Application: Aerosol particle size 415

24.4 Discussion 417

References 418

25 A Python package for Bayesian estimation using Markov chain Monte Carlo 421
Christopher M. Strickland, Robert J. Denham, Clair L. Alston and Kerrie L. Mengersen

25.1 Introduction 421

25.2 Bayesian analysis 423

25.2.1 MCMC methods and implementation 424

25.2.2 Normal linear Bayesian regression model 433

25.3 Empirical illustrations 437

25.3.1 Example 1: Linear regression model – variable selection and estimation 438

25.3.2 Example 2: Loglinear model 441

25.3.3 Example 3: First-order autoregressive regression 446

25.4 Using PyMCMC efficiently 451

25.4.1 Compiling code in Windows 455

25.5 PyMCMC interacting with R 457

25.6 Conclusions 458

25.7 Obtaining PyMCMC 459

References 459

Index 461