Assessment and Teaching of 21st Century Skills (eBook)

Preface 6
References 11
Contents 12
Part I: Project Method and Overview 14
Chapter 1: The ATC21S Method 16
Background 16
The Framework 20
Selecting the Skills 20
Expert Panels 20
Substantive Expert Panels 21
Learning Through Digital Networks 21
Collaborative Problem Solving 22
Problem Solving 22
Psychometric Panel 24
Teacher Panels 25
Education Context 25
Policy and Expectation 27
Research and Development 28
Formulating Hypotheses About Progressions 28
The Commissioned Task Developers 29
National Project Manager (NPM) Role 30
The Development Process 31
Drafting 33
Panelling 33
Cognitive Laboratories 34
Piloting 34
Trial 35
The Data Capture System 35
Mapping to the Blueprint 36
Coding 36
Conversion and Scoring 37
Calibrating 38
Estimating Student Ability Parameters and Determining the Levels 41
Reports 44
Teaching Twenty-First Century Skills 44
Conclusion 45
References 45
Part II: Conceptual Framework for Internet Based Collaborative Assessment Tasks 47
Chapter 2: A Framework for Teachable Collaborative Problem Solving Skills 48
Collaborative Problem Solving 49
Collaborative Problem Solving Processes 50
Collaborative Problem Solving Skills 52
Social Process Skills 53
Participation Skills 53
Perspective Taking Skills 55
Social Regulation Skills 56
Cognitive Process Skills 57
Task Regulation Skills 59
Learning and Knowledge Building Skills 61
Assessment of Collaborative Problem Solving Skills 63
Summary 64
References 65
Chapter 3: Assessment of Learning in Digital Networks 68
How to Assess Digital Learning 68
Learning in Networks: The Construct Map 69
The Four Strands 70
Learning in Networks: Three Scenarios 73
Arctic Trek 73
Webspiration 75
Second Language Chat 75
Sample Tasks from Arctic Trek 76
The Outcome Space for the Three Scenarios 80
Results from the Pilot Study 86
Conclusion and Next Steps 89
References 92
Part III: Delivery of Collaborative Tasks, Their Scoring, Calibration and Interpretation 93
Chapter 4: Collaborative Problem Solving Tasks 95
Introduction 95
Problem and Task Characteristics 96
Content-Free Collaborative Problem Solving Tasks 99
Laughing Clowns Task 100
Social Skill: Interaction 100
Social Skill: Audience Awareness 101
Cognitive Skill: Resource Management 101
Cognitive Skill: Relationships (Representing and Formulating) 101
Olive Oil Task 101
Social Skill: Interaction 102
Cognitive Skill: Cause and Effect 103
Cognitive Skill: Problem Analysis 103
Cognitive Skill: Solution 104
Content-Dependent Collaborative Problem Solving Tasks 104
Balance Beam Task 105
Social Skill: Responsiveness 106
Cognitive Skill: Systematicity 107
Cognitive Skill: Sets Goals 107
Game of 20 Task 107
Social Skill: Responsiveness 109
Social Skill: Responsibility Initiative 109
Cognitive Skill: Cause and Effect 109
Cognitive Skill: Reflects and Monitors (Testing Hypothesis) 110
Conclusion 111
Appendix: Collaborative Problem Solving Tasks 111
References 114
Chapter 5: Platforms for Delivery of Collaborative Tasks 115
Overview and Design Rationale 115
Delivery of CPS Tasks 116
Delivery of LDN-ICT Tasks 119
Lessons for an Integrated Collaborative Assessment Platform 121
Implications and Future 122
References 123
Chapter 6: Automatic Coding Procedures for Collaborative Problem Solving 124
Existing Approaches to Autoscoring 125
Design of Process Data Stream – Capturing and Identifying the Data 126
Defining the Indicators 128
Coding 129
Mapping 130
Scoring 133
Dichotomously Scored Indicators 133
Frequency-Based Indicators – Partial Credit Scoring 133
Evidence of Collaboration Within Indicators 135
Defining the Skills Progression 137
Challenges and Future Directions 138
References 140
Chapter 7: Task Characteristics and Calibration 142
Introduction 142
Concept and Construct Mapping 143
Specification and Blueprints 144
The Trial Samples 145
Task Calibration 145
Fit to the Model 148
Reliability Estimates 149
Establishing Validity Using Rasch Modelling 149
Calibration of the Tasks and Interpretation of the Data 150
Olive Oil 150
Laughing Clowns 151
Balance 153
Game of 20 155
Joint Calibration 157
Ability and Difficulty Estimates 157
One Dimension – One Parameter 159
Two Dimensions – One Parameter 161
Five Dimensions – One Parameter 161
Sets Analysis 164
Checking for Assessment Bias 165
Independence of Dyad Partners 165
Differential Item Functioning 167
Task Sets by Country 169
All Task Indicator Difficulties Compared with Australian Estimates 170
Interpreting and Reporting for Student Outcomes 173
Developing a Skills Progression 173
Assumptions 174
Standards and Benchmarks 175
Cognitive and Social Competence Levels 176
References 186
Part IV: Participating Countries in the Field Work 188
Chapter 8: Australia in the Context of the ATC21S Project 191
Context 192
ATC21S Development in Australia 193
Method 194
Task Concept Check 194
Cognitive Laboratory 196
Responses from the Task Concept Checks and Cognitive Laboratories 198
Pilot 200
Trials 200
Responses from the Pilot and Trials 202
Challenges 203
Conclusion 204
References 205
Chapter 9: Student and Teacher Responses to Collaborative Problem Solving and Learning Through Digital Networks in Singapore 206
Context 207
ATC21S Development in Singapore 208
Method 209
Responses to the ATC21S Tasks 210
Relevance to the Teaching, Learning and Assessment of 21st Century Competencies 210
Engagement with Tasks 211
Seeking Meaning in Content-Rich and Content-Free Tasks 212
Collaboration 213
Introducing Ambiguity in Tasks 215
Tools and Technical Issues 216
Challenges 217
Conclusion 217
References 218
Chapter 10: Introducing Assessment Tools for 21st Century Skills in Finland 220
Context 220
ATC21S Development in Finland 221
Method 222
Localization and Translation 222
Teacher and Student Responses to the ATC21S Tasks 224
Concept Checking 224
Cognitive Laboratory 226
Pilot and Trial Study 227
Challenges 228
Conclusions 231
References 231
Chapter 11: Case Study on the Implementation of ATC21S in the United States 233
Context 233
ATC21S Development in the U.S. 234
Method 236
Task Concept Review 236
Cognitive Laboratory 240
Pilot 242
Trials 243
Challenges 246
Conclusion 247
References 248
Chapter 12: The Adaptation and Contextualization of ATC21STM by Costa Rica 251
Context 251
ATC21S Development in Costa Rica 253
Localisation 254
The Arctic Trek 254
Webspiration (Poetry) 255
Method 257
Task Concept Checks, Cognitive Laboratories, Pilot Programs and Trials 257
Responses to ATC21S Tasks 258
Challenges 260
Conclusion 262
Chapter 13: ATC21S Trials of Collaborative Problem Solving Tasks in the Netherlands 263
Context 263
ATC21S Development in the Netherlands 264
Localisation 264
Method 266
Trials 266
Responses to the Trials 267
Challenges 268
Conclusion 269
Part V: Implementation at Classroom and System Levels 270
Chapter 14: Linking Developmental Progressions to Teaching 271
A Developmental Approach to Assessment and Learning 271
Using Developmental Frameworks to Describe and Understand Learning 274
Using a Developmental Model to Plan Teaching 275
Report Formats 276
Reviewing Student Progress 281
Generalising Intervention and Differentiating Instruction: Class Reports 283
Teaching 21st Century Skills 284
Teaching and Learning in Mixed Ability Classrooms 284
Application to Teaching and Learning Practice 285
Example: Collaborative Problem Solving – Cognitive Skills 285
Teaching and Learning Intention 285
Differentiation for Each Stage on the Cognitive Skill Progression 286
Variations for Students at Different Stages on the Social Skills Progression 289
Example: Learning in Networks – Building Intellectual Capital 291
Teaching and Learning Intention 291
Differentiation for Stages on the Skill Progression 292
Variations for Students at Different Stages on Other Skills Progressions 293
Summary 294
Definition of Terms 294
Freeze-Frame 294
Y Chart 294
Clock Buddies Strategy 295
References 296
Chapter 15: Policy Pathways for Twenty-First Century Skills 297
Organization of the Analysis 297
Curriculum in National Contexts 299
Incorporating 21st Century Skills into Curriculum 299
Curriculum Challenges 303
Teaching in National Contexts 303
Helping Teachers Enact 21st Century Skills 303
Challenges for Teaching 305
Assessing 21st Century Skills 305
Influences of International Assessment 305
National and State Assessment Programs 306
Local Assessments 307
Pathways for Adopting ATC21S 308
Adoption Challenges 309
ICT Access 309
Task Development 310
Level of Centralization 310
Conclusions 312
References 313