Assessing Model-Based Reasoning using Evidence- Centered Design (eBook)

Robert J. Mislevy, PhD, is Frederic M. Lord Chair in Measurement and Statistics at the Educational Testing Service in Princeton, New Jersey, and Professor Emeritus at the University of Maryland.Geneva D. Haertel, PhD, is Director of Assessment Research and Design at the Center for Technology in Learning, SRI International.Michelle M. Riconscente, PhD, is Director of Learning and Assessment at GlassLab in California.Daisy Wise Rutstein, PhD, is Education Researcher in the Education Division at SRI International.Cindy S. Ziker, PhD, MPH, is Senior Researcher for Assessment in the Education Division of the Center for Technology and Learning at SRI International.

Preface 7
Acknowledgements 10
Contents 11
About the Authors 13
1 Introduction 16
Abstract 16
2 Model-Based Reasoning 24
Abstract 24
2.1 Scientific Models 24
2.2 The Inquiry Cycle 27
2.3 Some Relevant Results from Psychology 29
2.3.1 Experiential Aspects of Model-Based Reasoning 29
2.3.2 Reflective Aspects of Model-Based Reasoning 30
2.3.3 Higher-Level Skills 31
2.3.4 Implications for Assessment Use Cases 32
3 Evidence-Centered Assessment Design 34
Abstract 34
3.1 Assessment Arguments 35
3.2 Design Patterns 37
4 Design Patterns for Model-Based Reasoning 40
Abstract 40
5 Model Formation 45
Abstract 45
5.1 Rationale, Focal KSAs, and Characteristic Task Features 45
5.2 Additional KSAs 50
5.3 Variable Task Features 53
5.4 Potential Work Products and Potential Observations 56
5.5 Considerations for Larger Investigations 60
5.6 Some Connections to Other Design Patterns 61
6 Model Use 63
Abstract 63
6.1 Rationale, Focal KSAs, and Characteristic Task Features 63
6.2 Additional KSAs 67
6.3 Variable Task Features 69
6.4 Potential Work Products and Potential Observations 69
6.5 Some Connections with Other Design Patterns 71
7 Model Elaboration 72
Abstract 72
7.1 Rationale, Focal KSAs, and Characteristic Task Features 72
7.2 Additional KSAs 74
7.3 Variable Task Features 74
7.4 Potential Work Products and Potential Observations 76
7.5 Some Connections with Other Design Patterns 77
8 Model Articulation 78
Abstract 78
8.1 Rationale, Focal KSAs, and Characteristic Task Features 79
8.2 Additional KSAs 80
8.3 Variable Task Features 80
8.4 Potential Work Products and Potential Observations 81
8.5 Some Connections with Other Design Patterns 82
9 Model Evaluation 84
Abstract 84
9.1 Rationale, Focal KSAs, and Characteristic Task Features 84
9.2 Additional KSAs 89
9.3 Variable Task Features 90
9.4 Potential Work Products and Potential Observations 91
9.5 Some Connections with Other Design Patterns 92
10 Model Revision 93
Abstract 93
10.1 Rationale, Focal KSAs, and Characteristic Task Features 93
10.2 Additional KSAs 96
10.3 Variable Task Features 97
10.4 Potential Work Products and Potential Observations 98
10.5 Some Connections with Other Design Patterns 98
11 Model-Based Inquiry 100
Abstract 100
11.1 Rationale, Focal KSAs, and Characteristic Task Features 101
11.2 Additional KSAs 103
11.3 Variable Task Features 103
11.4 Potential Work Products and Potential Observations 105
11.5 Some Connections with Other Design Patterns 108
12 Conclusion 109
Abstract 109
12.1 Standards-Based Assessment 109
12.2 Classroom Assessment 110
12.3 Large-Scale Accountability Testing 111
12.4 Simulation- and Game-Based Assessment 112
12.5 Closing Comments 113
Appendix: Summary Form of Design Patterns for Model-Based Reasoning 114
References 129
Index 136