Evaluating Children's Interactive Products (eBook)

FRONT COVER 1
EVALUATING CHILDREN'S INTERACTIVE PRODUCTS 4
COPYRIGHT PAGE 5
CONTENTS 6
PREFACE 18
ABOUT THE AUTHORS 24
PART 1 CHILDREN AND TECHNOLOGY 28
CHAPTER 1 WHAT IS A CHILD? 29
Age and Children 30
Learning about Children 31
Theories of Child Development 31
Perspectives on Child Development 32
Biological 32
Psychodynamic 33
Learning 34
Cognitive-Developmental 35
Sociocultural 36
Summary 37
Typical Stages of Development 38
Physical 38
Socioemotional 39
Cognitive 40
Child Development and the Evaluation of Interactive Products 41
The Temperament of Children 42
Temperamental Dimensions 42
Reducing the Effects of Temperament 44
Summary 44
Further Reading 45
CHAPTER 2 CHILDREN AND INTERACTIVE TECHNOLOGY 47
Interactive Products 48
What Is Interactivity? 48
Measuring Interaction 49
Interactive Products for Children 50
Entertainment Products 51
Education Products 52
Enabling Products 52
How Children Use Interactive Products 54
The PLU Model 55
Interactive Technology and Evaluation Studies 56
How Good Is Interactive Technology for Children? 58
Gender and Technology 58
Summary 60
Further Reading 60
CHAPTER 3 THE INTERACTIVE PRODUCT LIFECYCLE 61
Interaction Design and Evaluation 62
The Product Lifecycle 63
The Waterfall Model 63
The Usability Engineering Lifecycle Model 65
Using Lifecycle Models 67
The Place of Evaluation in the Lifecycle 68
Using Prototypes in Evaluation 68
Medium Used 69
Scope 69
Fit in the Lifecycle 70
Purpose 71
Involving Children in Design and Evaluation 72
Summary 75
Further Reading 75
PART 2 EVALUATING WITH AND FOR CHILDREN 76
CHAPTER 4 ETHICAL PRACTICE IN EVALUATIONS 77
Ethical Principles, Approaches, and Codes 78
Cultural Issues 80
Ethical Codes 81
Ethics and the Law 81
Safety and Risk Assessment 82
Consent 83
Consenting to Take Part in the Study 83
Deception 86
Covert Observation 86
Consent for Photographs and Audio and Video Recordings 87
Inducements for Taking Part 89
Ethical Selection of Participants 90
Privacy 91
Getting Advice and Permission 91
Further Reading 93
CHAPTER 5 PLANNING THE EVALUATION STUDY 95
Defining the Purpose of the Evaluation 96
Diagnostic Evaluations 96
Formative and Summative Evaluations 97
Exploratory, Measurement, and Experimental Studies 97
Evaluation Criteria 98
Choosing Evaluation Methods 103
Reliability of Evaluation Results 103
Field versus Lab: More Than Just a Location! 104
Control and Realism 105
Evaluating Products with Children in the Field 108
Goals and Testing Context 109
The Evaluation Plan—Why Make One? 110
Context Definition 110
Evaluation Goal 110
Evaluation Method 112
Location and Profile 112
Summary 115
Further Reading 116
CHAPTER 6 BEFORE THE EVALUATION 117
Deciding on People and Places 119
Confirming a Location and a Time 119
Selecting Child Participants 122
Selecting the Evaluation Team 124
Preparing Technology and Evaluation Instruments 125
Preparing the Technology 125
Designing Test Tasks 125
Evaluation Instruments and Instructions 130
Planning the Day 132
Space 132
Managing Children, Tasks, and Technology 134
Training Evaluators and Pilot Testing 135
Communicating the Details 136
Next Stages 137
CHAPTER 7 DURING THE EVALUATION 139
Arrival and Setup 140
Preparing the Evaluation Team 141
Setting up the Space 141
Introductions and Instructions 142
Children in Adult Places 142
Adults in Children's Places 145
Giving Instructions 146
Managing Time and Space 148
Working with Schoolchildren 149
Monitoring the Tasks 149
Dealing with Problems on the Day 150
Closing the Evaluation 152
Leaving the Area Tidy 152
Saying "Thank You!" 152
CHAPTER 8 AFTER THE EVALUATION 155
Getting the Data into Shape 156
Coding and Analyzing the Data 158
Coding and Analyzing Quantitative Data 158
Presenting Summaries of Quantitative Data 159
Analyzing Qualitative Data 161
Reporting Results 163
Know Your Audience 163
Structure and Style 164
Oral Presentation of Results 168
Care of Data 169
How Long Should Data Be Kept? 170
Reflecting on the Process 171
Summary 172
PART 3 METHODS OF EVALUATION 174
CHAPTER 9 RECORDING AND LOGGING 175
Automated Logging 176
Recording the Data 177
Software for Logging 177
Practicalities 177
Video and Audio 178
The Technology 178
Setup and Use in Usability Labs and Elsewhere 180
Gaze Tracking 184
Gaze-Tracking Measurements 184
Gaze-Tracking Technology 186
Hardware for Participants 187
Setup and Calibration with Children 189
Using Gaze Tracking with Children 189
Summary 190
Further Reading 190
CHAPTER 10 OBSERVATION METHODS 191
Types of Observation 192
Realism in Observation Studies 193
Participant Observation 193
Passive and Naturalistic Observation 194
Structured Observation 196
Determine the Focus 197
Develop Guides and Forms 200
Recruit and Train Observers 203
Carry out the Observation 204
Analyze and Interpret Findings 206
Using Standard Coding Schemes 208
Summary 210
CHAPTER 11 VERBALIZATION METHODS 213
Types of Verbalization Methods 214
Think-Aloud 215
Variants of Think-Aloud 222
The Picture Cards Method 223
Other Variants of Think-Aloud 226
Dialogue between the Administrator and the Testers 229
Active Intervention 229
Post-Task Interview 230
Robotic Intervention 230
Interaction between Testers 233
Constructive Interaction 233
Peer Tutoring 234
Methodological Issues of Verbalization Methods 237
Advantages and Disadvantages 237
Performance 238
Which Children Should You Test? 241
Analyzing Verbal Protocols 241
Summary 242
Further Reading 243
CHAPTER 12 THE WIZARD OF OZ METHOD 245
Wizard of Oz Studies 246
Variability 248
Studies with Children 250
When to Use a WOz Study 251
A Walkthrough of a WOz Evaluation 253
The Wizard Interface 253
Overcoming Problems 255
Managing the Evaluation Study 255
Stages in Planning a Study 256
Problems 257
Children as Users of Technology 257
The Wizard Is a Human Being 258
Summary 259
Further Reading 259
CHAPTER 13 SURVEY METHODS 261
What Is a Survey? 262
The Purpose of Surveys in Evaluation 263
Why Use a Survey Method? 264
Designing a Survey 264
Designing the Questions 267
What Can Go Wrong? 268
Satisficing and Optimizing 270
Specific Question Formats 270
Language Effects 271
Carrying out the Survey 272
Minimizing Suggestibility 273
Reliability of Findings 275
The Fun Toolkit 277
Guidelines for Using the Fun Toolkit 281
Summary 281
Further Reading 282
CHAPTER 14 DIARIES 283
Field Evaluation 284
Using Diaries to Evaluate Interactive Products with Children 285
Design the Study 286
Prepare the Diary Questions and Materials 288
Monitor and Encourage Diary Keeping 289
Perform the Debriefing Interview 290
The Parent Evaluator Method 291
Procedure 291
Structuring Diaries 292
How Good Is the Method? 293
More Guidelines for Diary Studies 294
Summary 294
Further Reading 294
CHAPTER 15 INSPECTION METHODS 295
Heuristic Evaluation 297
The Process 297
Nielsen's Usability Heuristics 299
Problem Reporting 300
Problems with Heuristic Evaluation of Children's Products 301
Using the Heuristic Evaluation Method with Other Heuristics 302
Heuristics for Evaluating Fun 302
Heuristics for Evaluating Games 304
Heuristics for Evaluating Web Sites 306
Using Children as Evaluators 307
The SEEM Method 307
Persona-Based Evaluation 311
How Good Are Inspection Methods? 318
Summary 320
Further Reading 321
PART 4 CASE STUDIES 322
CHAPTER 16 CASE STUDY 1: GAME-CONTROLLING GESTURES IN INTERACTIVE GAMES 323
Finding a Suitable Evaluation Method 324
The Study 326
Preparations 326
Interactive Prototypes 327
Test Setup 328
Participants 329
Test Space Design 329
Procedure 330
Children's Movement Analysis 330
Preliminary Video Analysis 330
Experimenting with Movement Description Methods 332
Results from the Study 334
Intuitive Movements for the Swimming Game 334
Commentary on the Study 336
Ethics 336
Test Arrangements 338
Pilot Analysis of the Test Data 338
Summary 339
CHAPTER 17 CASE STUDY 2: EMBEDDING EVALUATION IN THE DESIGN OF A PERVASIVE GAME CONCEPT 341
The Design Project Context 342
Children and the Design of Camelot 342
The Mission from Mars Method 343
The Interview 343
Paper Prototypes with Observations and Picture Card Interviews 347
Paper Prototype Evaluations 347
Reflections on Concept Testing 349
Evaluating Interaction Styles with Peer Tutoring 349
The Camelot Game 349
Peer Tutoring Evaluations 350
Surveying Form Preferences 351
The Paired Comparison Test 352
Evaluation Summary of the Camelot Design Concept 352
Play Testing of the Camelot Prototype 353
Summary 355
CHAPTER 18 CASE STUDY 3: USING SURVEY METHODS AND EFFICIENCY METRICS 357
The Study 358
Training 360
Test Task Design 361
Constructs Being Evaluated 362
Results 366
Commentary 367
Efficiency 367
Satisfaction 368
Summary 368
REFERENCES 370
INDEX 382
A 382
B 383
C 383
D 384
E 385
F 386
G 387
H 387
I 388
J 389
K 389
L 389
M 389
N 390
O 390
P 390
Q 392
R 392
S 392
T 394
U 395
V 395
W 395
Y 396