Linking and Aligning Scores and Scales (eBook)

Dedication 6
Foreword 7
Preface 9
List of Contributors 19
1 Overview 21
Part 1: Foundations 23
2 A Framework and History for Score Linking 24
2.1. Introduction 24
2.2. Predicting 25
2.4. Equating: Same Construct and the Same Intended Difficulty and Reliability 39
2.5. A Brief Note on the Theory of Equating 48
3 Data Collection Designs and Linking Procedures 50
3.1. Introduction 50
3.2. Features of Testing Situations 51
3.3. Types of Linking Considered 52
3.4. Linking Functions and Features of Testing Situations 54
3.5. Linking Designs 56
3.6. Linking Procedures 68
3.7. Summary and Conclusions 73
Part 2: Equating 75
4 Equating: Best Practices and Challenges to Best Practices 76
4.1. Equating 76
4.2. Best Practices 78
4.3. Challenges to Best Practices 80
4.4. Discussion 87
4.5. Summary 88
5 Practical Problems in Equating Test Scores: A Practitioner’s Perspective 90
5.1. Introduction 90
5.2 The Nonequivalent Groups Anchor Test Design 92
5.3. Characteristics of the New and Old Forms 93
5.4. Characteristics of the Groups Used for Equating 96
5.5. Characteristics of the Anchor Test (Common Items) 101
5.6. Conclusions 104
6 Potential Solutions to Practical Equating Issues 106
6.1. Introduction 106
6.2. Observed-Score Equating Methods 108
6.3. Addressing the Fairness Issue: Population Invariance of Equating Functions 109
6.4. Addressing the Small-Samples Issue: Synthetic Linking Functions 113
6.5. Addressing Differences in Ability in the Two Populations of the NEAT Design 116
6.6. Addressing the Stability of Equating Results: Kernel Equating and Applications 118
6.7. Discussion 122
Part 3: Tests in Transition 124
7 Score Linking Issues Related to Test Content Changes 125
7.1. Introduction 125
7.2. Major Linking Issues for Tests in Transition 127
7.3. Considerations for Data Collection Design 133
7.4. Equatability Analyses 138
7.5. Discussion 149
8 Linking Scores Derived Under Different Modes of Test Administration 151
8.1. Introduction 151
8.2. Background 152
8.3. Comparability Issues Involving Scores from Computer- Based and Paper- and- Pencil Tests 154
8.4. Mode of Presentation Linking Designs 157
8.5. Random Groups Design 158
8.6. Single Group Counterbalanced Test Design 163
8.7. Anchor Test: Nonequivalent Groups Design 170
8.8. Summary 174
9 Tests in Transition: Discussion and Synthesis 176
9.1. The Liu and Walker Chapter on Test Content Changes 177
9.2. Eignor Chapter on Mode of Administration 183
9.3. Additional Perspectives 186
Part 4: Concordance 191
10 Sizing Up Linkages 192
10.1. Introduction 192
10.2. What Makes a Linking an Equating? 194
10.3. Why Is Reliability Important? 196
10.4. Uncertainty Reduction 197
10.5. Population Invariance of Linking Functions 201
10.6. Sizing Up Linkages 204
10.7. Summary 210
10.8. Postscript 211
11 Concordance: The Good, the Bad, and the Ugly 212
11.1 Background 212
11.2. Definitions 214
11.3. The Concordance Process 215
11.4. The Concordance Dilemma 224
11.5. Conclusions 228
12 Some Further Thoughts on Concordance 230
12.1. Four Common Score Uses in College Admission 230
12.2. Other Thoughts on Concordance Tables 241
12.3. Coastal Aspirations and Midland Acquisitions 242
12.4. Conclusions 242
12.5. Postscript 243
Part 5: Vertical Scaling 244
13 Practical Issues in Vertical Scaling 245
13.1. Introduction 245
13.2. Conceptual Issues 247
13.3. Technical Issues 252
13.4. Implementation Issues 257
13.5. Scale Maintenance Issues 260
13.6. Other Issues 262
13.7. Summary 262
14 Methods and Models for Vertical Scaling 264
14.1. Introduction 264
14.2. General Modeling Issues in Vertical Scaling Contexts 265
14.3. A Multidimensional, Multigroup IRT Model for Vertical Scaling 271
14.4. Discussion 282
15 Vertical Scaling and No Child Left Behind 284
15.1. Comments on the Other Vertical Scaling Chapters 284
15.2. Vertical Scales: An Historical Perspective 285
15.3. The NCLB Era 289
15.4. Summary 293
Part 6: Linking Group Assessments to Individual Assessments 295
16 Linking Assessments Based on Aggregate Reporting: Background and Issues 296
16.1. Linking Methods 298
16.2. The Seeds Are Sown: Linkages of NAEP with IAEP and the ASVAB 300
16.3. State Linkages with NAEP 304
16.5. More Recent Linkages with NAEP 313
16.6. Problems Requiring Further Research 315
16.7. Conclusions 319
17 An Enhanced Method for Mapping State Standards onto the NAEP Scale 322
17.1. Introduction 322
17.2. Outline of the Methodology 325
17.3. Details of the Methodology 327
17.4. Estimation of Variances 333
17.5. Results 337
17.6. Mapping the NAEP Achievement Standards onto a State Test Scale 340
17.7. Conclusions and Recommendations 342
18 Using Aggregate-Level Linkages for Estimation and Validation: Comments on Thissen and Braun & Qian
18.1. Comments on the Thissen Chapter 349
18.2. Comments on Braun and Qian 351
18.3. More on Failures of Invariance over Time 357
18.4. Conclusion 362
19 Postscript 363
19.1. The Descent of Linking 363
19.2. Extreme Linkages 364
19.3. IRT: Tool Versus Theory 365
19.4. Future Trends 365
19.5. Closing Comments 366
References 368
Author Index 389
Subject Index 394