Adaptive Tests of Significance Using Permutations of Residuals with R and SAS

Thomas W. O'gorman, PhD, is Associate Professor in the Department of Mathematical Sciences at Northern Illinois University. Dr. O'Gorman's current research focuses on the analysis of adaptive methods for performing statistical tests and confidence intervals.

Preface xv 1 Introduction 1

1.1 Why Use Adaptive Tests? 1

1.2 A Brief History of Adaptive Tests 2

1.3 The Adaptive Test of Hogg, Fisher, and Randies 5

1.4 Limitations of Rank-Based Tests 8

1.5 The Adaptive Weighted Least Squares Approach 9

1.6 Development of the Adaptive WLS Test 12

2 Smoothing Methods and Normalizing Transformations 15

2.1 Traditional Estimators of the Median and the Interquartile Range 15

2.2 Percentile Estimators that Use the Smooth Cumulative Distribution Function 16

2.3 Estimating the Bandwidth 21

2.4 Normalizing Transformations 23

2.5 The Weighting Algorithm 27

2.6 Computing the Bandwidth 30

2.7 Examples of Transformed Data 37

3 A Two-Sample Adaptive Test 43

3.1 A Two-Sample Model 44

3.2 Computing the Adaptive Weights 45

3.3 The Test Statistics for Adaptive Tests 47

3.4 Permutation Methods for Two-Sample Tests 50

3.5 An Example of a Two-Sample Test 54

3.6 R Code for the Two-Sample Test 56

3.7 Level of Significance of the Adaptive Test 61

3.8 Power of the Adaptive Test 63

3.9 Sample Size Estimation 65

3.10 A SAS Macro for the Adaptive Test 68

3.11 Modifications for One-Tailed Tests 70

3.12 Justification of the Weighting Method 70

3.13 Comments on the Adaptive Two-sample Test 71

4 Permutation Tests with Linear Models 75

4.1 Introduction 75

4.2 Notation 76

4.3 Permutations with Blocking 77

4.4 Linear Models in Matrix Form 77

4.5 Permutation Methods 78

4.6 Permutation Test Statistics 81

4.7 An Important Rule of Test Construction 82

4.8 A Permutation Algorithm 82

4.9 A Performance Comparison of the Permutation Methods 83

4.10 Discussion 84

5 An Adaptive Test for a Subset of Coefficients 87

5.1 The General Adaptive Testing Method 87

5.2 Simple Linear Regression 91

5.3 An Example of a Simple Linear Regression 93

5.4 Multiple Linear Regression 96

5.5 An Example of a Test in Multiple Regression 100

5.6 Conclusions 105

6 More Applications of Adaptive Tests 111

6.1 The Completely Randomized Design 111

6.2 Tests for Randomized Complete Block Designs 120

6.3 Adaptive Tests for Two-way Designs 127

6.4 Dealing with Unequal Variances 134

6.5 Extensions to More Complex Designs 140

7 The Adaptive Analysis of Paired Data 149

7.1 Introduction 149

7.2 The Adaptive Test of Miao and Gastwirth 151

7.3 An Adaptive Weighted Least Squares Test 153

7.4 An Example Using Paired Data 160

7.5 Simulation Study 161

7.6 Sample Size Estimation 163

7.7 Discussion of Tests for Paired Data 165

8 Multicenter and Cross-Over Trials 169

8.1 Tests in Multicenter Clinical Trials 170

8.2 Adaptive Analysis of Cross-over Trials 176

9 Adaptive Multivariate Tests 191

9.1 The Traditional Likelihood Ratio Test 191

9.2 An Adaptive Multivariate Test 192

9.3 An Example with Two Dependent Variables 196

9.4 Performance of the Adaptive Test 199

9.5 Conclusions for Multivariate Tests 203

10 Analysis of Repeated Measures Data 207

10.1 Introduction 207

10.2 The Multivariate LR Test 209

10.3 The Adaptive Test 209

10.4 The Mixed Model Test 210

10.5 Two-Sample Tests 211

10.6 Two-Sample Tests for Parallelism 212

10.7 Two-Sample Tests for Group Effect 219

10.8 An Example of Repeated Measures Data 223

10.9 Dealing with Missing Data 227

10.10 Conclusions and Recommendations 229

11 Rank-Based Tests of Significance 235

11.1 The Quest for Power 235

11.2 Two-Sample Rank Tests 236

11.3 The HFR Test 242

11.4 Significance Level of Adaptive Tests 243

11.5 Biining's Adaptive Test for Location 244

11.6 An Adaptive Test for Location and Scale 245

11.7 Other Adaptive Rank Tests 247

11.8 Maximum Test 248

11.9 Discussion 249

12 Adaptive Confidence Intervals and Estimates 253

12.1 The Relationship Between Tests and Confidence Intervals 253

12.2 The Iterative Procedure of Garthwaite 254

12.3 Confidence Interval for a Difference 259

12.4 A 95% Confidence Interval for Slope 263

12.5 A General Formula for Confidence Limits 264

12.6 Computing a Confidence Interval Using R 266

12.7 Computing a 95% Confidence Interval Using SAS 268

12.8 Adaptive Estimation 268

12.9 Adaptive Estimation of the Difference Between Two Population Means 271

12.10 Adaptive Estimation of a Slope in a Multiple Regression Model 272

12.11 Computing an Adaptive Estimate Using R 274

12.12 Computing an Adaptive Estimate Using SAS 278

12.13 Discussion 278

Exercises 279

Appendix A: R Code for Univariate Adaptive Tests 283

Appendix B: SAS Macro for Adaptive Tests 287

Appendix C: SAS Macro for Multiple Comparisons Procedures 299

Appendix D: R Code for Adaptive Tests with Blocking Factors 303

Appendix E: R Code for Adaptive Test with Paired Data 305

Appendix F: SAS Macro for Adaptive Test with Paired Data 309

Appendix G: R Code for Multivariate Adaptive Tests 313

Appendix H: R Code for Confidence Intervals and Estimates 317

Appendix I: SAS Macro for Confidence Intervals 321

Appendix J: SAS Macro for Estimates 329

References 333

Index 341