Wiley Handbook of Cognition and Assessment (eBook)

André A. Rupp is Research Director at Educational Testing Service (ETS) in Princeton, NJ, where he works with teams that conduct comprehensive evaluation work for mature and emerging automated scoring systems. His research has focused on applications of principled assessment design frameworks in innovative assessment contexts as well as translating the statistical complexities of diagnostic measurement models into practical guidelines for applied specialists. Through dissemination and professional development efforts he is deeply dedicated to helping interdisciplinary teams navigate the complicated trade-offs between scientific, educational, political, and financial drivers of decision-making in order to help shape best methodological practices for evidentiary reasoning for complex assessment design and deployment lifecycles. He is co-author of Diagnostic Measurement: Theory, Methods, and Applications (2010). Jacqueline P. Leighton is Professor and Chair of Educational Psychology at the University of Alberta, Canada. She is past Director of the University of Alberta's Centre for Research in Applied Measurement and Evaluation (CRAME). As a registered psychologist with the College of Alberta Psychologists, her research is focused on measuring the cognitive and socio-emotional processes underlying learning and assessment outcomes, including cognitive diagnostic assessment and feedback delivery and uptake. She has published in a variety of educational measurement journals and is past editor of Educational Measurement: Issues and Practice. She is co-author of The Learning Sciences in Educational Assessment (2011) and Cognitive Diagnostic Assessment for Education: Theory and Applications (2007) and co-editor of The Nature of Reasoning (2004).

Notes on Contributors ix

Foreword xix

Acknowledgements xxi

1 Introduction to Handbook 1
André A. Rupp and Jacqueline P. Leighton

Part I Frameworks 13

2 The Role of Theories of Learning and Cognition in Assessment Design and Development 15
Paul D. Nichols, Jennifer L. Kobrin, Emily Lai, and James Koepfler

3 Principled Approaches to Assessment Design, Development, and Implementation 41
Steve Ferrara, Emily Lai, Amy Reilly, and Paul D. Nichols

4 Developing and Validating Cognitive Models in Assessment 75
Madeleine Keehner, Joanna S. Gorin, Gary Feng, and Irvin R. Katz

5 An Integrative Framework for Construct Validity 102
Susan Embretson

6 The Role of Cognitive Models in Automatic Item Generation 124
Mark J. Gierl and Hollis Lai

7 Social Models of Learning and Assessment 146
William R. Penuel and Lorrie A. Shepard

8 Socio?]emotional and Self?]management Variables in Learning and Assessment 174
Patrick C. Kyllonen

9 Understanding and Improving Accessibility for Special Populations 198
Leanne R. Ketterlin?]Geller

10 Automated Scoring with Validity in Mind 226
Isaac I. Bejar, Robert J. Mislevy, and Mo Zhang

Part II Methodologies 247

11 Explanatory Item Response Models 249
Paul De Boeck, Sun?]Joo Cho, and Mark Wilson

12 Longitudinal Models for Repeated Measures Data 267
Jeffrey R. Harring and Ari Houser

13 Diagnostic Classification Models 297
Laine Bradshaw

14 Bayesian Networks 328
José P. González?]Brenes, John T. Behrens, Robert J. Mislevy, Roy Levy,and Kristen E. DiCerbo

15 The Rule Space and Attribute Hierarchy Methods 354
Ying Cui, Mark J. Gierl, and Qi Guo

16 Educational Data Mining and Learning Analytics 379
Ryan S. Baker, Taylor Martin, and Lisa M. Rossi

Part III Applications 397

17 Large?]Scale Standards?]Based Assessments of Educational Achievement 399
Kristen Huff, Zachary Warner, and Jason Schweid

18 Educational Survey Assessments 427
Andreas Oranje, Madeleine Keehner, Hilary Persky, Gabrielle Cayton?]Hodges, and Gary Feng

19 Professional Certification and Licensure Examinations 446
Richard M. Luecht

20 The In?]Task Assessment Framework for Behavioral Data 472
Deirdre Kerr, Jessica J. Andrews, and Robert J. Mislevy

21 Digital Assessment Environments for Scientific Inquiry Practices 508
Janice D. Gobert and Michael A. Sao Pedro

22 Assessing and Supporting Hard?]to?]Measure Constructs in Video Games 535
Valerie Shute and Lubin Wang

23 Conversation?]Based Assessment 563
G. Tanner Jackson and Diego Zapata?]Rivera

24 Conclusion to Handbook 580
Jacqueline P. Leighton and André A. Rupp

Glossary 588

Index 603