Handbook of Usability Testing

Jeffrey Rubin, author of the first edition of this book, has more than 30 years of experience as a human factors and usability research consultant and lecturer. A pioneer in the field of usability testing, he has consulted for leading companies worldwide. Dana Chisnell is an independent user researcher and usability consultant. She has done usability, user interface design, and technical communications consulting and development since 1982. Companion Web site Go to www.wiley.com/go/usabilitytesting to find additional case examples, samples, and templates

Acknowledgments xi

Foreword xxix

Preface to the Second Edition xxxiii

Part One Usability Testing: An Overview

Chapter 1 What Makes Something Usable? 3

What Do We Mean by ‘‘Usable’’? 4

What Makes Something Less Usable? 6

WhatMakes Products MoreUsable? 12

What Are Techniques for Building in Usability? 16

Chapter 2 What Is Usability Testing? 21

Why Test? Goals of Testing 21

Basics of the Methodology 23

Chapter 3 When Should You Test? 27

Our Types of Tests: An Overview 27

Exploratory or Formative Study 29

Assessment or Summative Test 34

Validation or Verification Test 35

Overview of the Methodology 36

Comparison Test 37

Iterative Testing: Test Types through the Lifecycle 39

Chapter 4 Skills for Test Moderators 45

Characteristics of a Good Test Moderator 48

Getting the Most out of Your Participants 52

Troubleshooting Typical Moderating Problems 56

How to Improve Your Session-Moderating Skills 58

Part Two The Process for Conducting a Test

Chapter 5 Develop the Test Plan 65

Why Create a Test Plan? 65

The Parts of a Test Plan 67

Sample Test Plan 91

Chapter 6 Set Up a Testing Environment 93

Decide on a Location and Space 94

Recommended Testing Environment: Minimalist Portable Lab 110

Gather and Check Equipment, Artifacts, and Tools 111

Identify Co-Researchers, Assistants, and Observers 112

Chapter 7 Find and Select Participants 115

Characterize Users 115

Define the Criteria for Each User Group 119

Determine the Number of Participants to Test 125

Write the Screening Questionnaire 126

Find Sources of Participants 131

Screen and Select Participants 143

Schedule and Confirm Participants 148

Chapter 8 Prepare Test Materials 153

Guidelines for Observers 154

Orientation Script 155

Background Questionnaire 162

Data Collection Tools 165

Nondisclosures, Consent Forms, and Recording Waivers 173

Pre-Test Questionnaires and Interviews 174

Prototypes or Products to Test 181

Task Scenarios 182

Optional Training Materials 187

Use the Research Questions(s) from the Test Plan as the Basis for Your Content 193

Common Question Formats 197

Debriefing Guide 199

Chapter 9 Conduct the Test Sessions 201

Guidelines for Moderating Test Sessions 202

Checklists for Getting Ready 213

When to Intervene 225

What Not to Say to Participants 227

Chapter 10 Debrief the Participant and Observers 229

Why Review with Participants and Observers? 229

Techniques for Reviewing with Participants 230

Where to Hold the Participant Debriefing Session 231

Basic Debriefing Guidelines 231

Advanced Debriefing Guidelines and Techniques 235

Reviewing and Reaching Consensus with Observers 241

Chapter 11 Analyze Data and Observations 245

Compile Data 246

Summarize Data 249

Analyze Data 258

Chapter 12 Report Findings and Recommendations 269

What Is a Finding? 269

Shape the Findings 269

Draft the Report 271

Develop Recommendations 277

Refine the Report Format 283

Create a Highlights Video or Presentation 283

Part Three Advanced Techniques

Chapter 13 Variations on the Basic Method 293

Who? Testing with Special Populations 293

What: Prototypes versus Real Products 299

How? Techniques for Monitored Tests 302

Where? Testing Outside a Lab 309

Self-Reporting (Surveys, Diary Studies) 313

Chapter 14 Expanding from Usability Testing to Designing the User Experience 315

Stealth Mode: Establish Value 316

Build on Successes 322

Formalize Processes and Practices 323

Expand UCD throughout the Organization 328

Afterword 333

Index 335