Agile Testing

Lisa Crispin is dedicated to helping agile teams and testers discover good ways to deliver the best possible product. She specializes in showing testers and agile teams how testers can add value and how to guide development with business-facing tests. Since 2003, she’s been a tester on a Scrum/XP team at ePlan Services, Inc., and frequently leads tutorials and workshops on agile testing at conferences. Lisa regularly contributes articles about agile testing to publications such as Better Software magazine, I EEE Software, and Methods and Tools. Lisa also coauthored Testing Extreme Programming (Addison-Wesley, 2002) with Tip House. Janet Gregory is the founder of DragonFire, Inc., an agile quality process consultancy and training firm. Her passion is helping teams build quality systems. Since 1998, she has worked as a coach and tester introducing agile practices into both large and small companies. Her focus is working with business users and testers to understand their role in agile projects. Janet is a frequent speaker at agile and testing software conferences, and she is a major contributor to the North American agile testing community.

Foreword by Mike Cohn      xxiii
Foreword by Brian Marick      xxv
Preface      xxvii
Acknowledgments      xxxvii
About the Authors      xli

Part I: Introduction      1
Chapter 1: What Is Agile Testing, Anyway?      3
Agile Values 3
What Do We Mean by “Agile Testing”? 4
A Little Context for Roles and Activities on an Agile Team 7
How Is Agile Testing Different? 9
Whole-Team Approach 15
Summary 17

Chapter 2: Ten Principles for Agile Testers      19
What’s an Agile Tester? 19
The Agile Testing Mind-Set 20
Applying Agile Principles and Values 21
Adding Value 31
Summary 33

Part II: Organizational Challenges      35
Chapter 3: Cultural Challenges      37
Organizational Culture 37
Barriers to Successful Agile Adoption by Test/QA Teams 44
Introducing Change 49
Management Expectations 52
Change Doesn’t Come Easy 56
Summary 58

Chapter 4: Team Logistics      59
Team Structure 59
Physical Logistics 65
Resources 66
Building a Team 69
Summary 71

Chapter 5: Transitioning Typical Processes      73
Seeking Lightweight Processes 73
Metrics 74
Defect Tracking 79
Test Planning 86
Existing Processes and Models 88
Summary 93

Part III: The Agile Testing Quadrants      95
Chapter 6: The Purpose of Testing      97
The Agile Testing Quadrants 97
Knowing When a Story Is Done 104
Managing Technical Debt 106
Testing in Context 106
Summary 108

Chapter 7: Technology-Facing Tests that Support the Team       109
An Agile Testing Foundation 109
Why Write and Execute These Tests? 112
Where Do Technology-Facing Tests Stop? 119
What If the Team Doesn’t Do These Tests? 121
Toolkit 123
Summary 127

Chapter 8: Business-Facing Tests that Support the Team      129
Driving Development with Business-Facing Tests 129
The Requirements Quandary 132
Thin Slices, Small Chunks 144
How Do We Know We’re Done? 146
Tests Mitigate Risk 147
Testability and Automation 149
Summary 150

Chapter 9: Toolkit for Business-Facing Tests that Support the Team       153
Business-Facing Test Tool Strategy 153
Tools to Elicit Examples and Requirements 155
Tools for Automating Tests Based on Examples 164
Strategies for Writing Tests 177
Testability 183
Test Management 186
Summary 186

Chapter 10: Business-Facing Tests that Critique the Product      189
Introduction to Quadrant 3 190
Demonstrations 191
Scenario Testing 192
Exploratory Testing 195
Usability Testing 202
Behind the GUI 204
Testing Documents and Documentation 207
Tools to Assist with Exploratory Testing 210
Summary 214

Chapter 11: Critiquing the Product Using Technology-Facing Tests       217
Introduction to Quadrant 4 217
Who Does It? 220
When Do You Do It? 222
“ility” Testing 223
Performance, Load, Stress, and Scalability Testing 233
Summary 238

Chapter 12: Summary of Testing Quadrants      241
Review of the Testing Quadrants 241
A System Test Example 242
Tests Driving Development 244
Automation 245
Critiquing the Product with Business-Facing Tests 248
Documentation 251
Using the Agile Testing Quadrants 252
Summary 253

Part IV: Automation      255
Chapter 13: Why We Want to Automate Tests and What Holds Us Back      257
Why Automate? 258
Barriers to Automation—Things that Get in the Way 264
Can We Overcome These Barriers? 270
Summary 271

Chapter 14: An Agile Test Automation Strategy        273
An Agile Approach to Test Automation 274
What Can We Automate? 279
What Shouldn’t We Automate? 285
What Might Be Hard to Automate? 287
Developing an Automation Strategy—Where Do We Start? 288
Applying Agile Principles to Test Automation 298
Supplying Data for Tests 304
Evaluating Automation Tools 311
Implementing Automation 316
Managing Automated Tests 319
Go Get Started 324
Summary 324

Part V: An Iteration in the Life of a Tester      327
Chapter 15: Tester Activities in Release or Theme Planning      329
The Purpose of Release Planning 330
Sizing 332
Prioritizing 338
What’s in Scope? 340
Test Planning 345
Test Plan Alternatives 350
Preparing for Visibility 354
Summary 366

Chapter 16: Hit the Ground Running       369
Be Proactive 369
Advance Clarity 373
Examples 378
Test Strategies 380
Prioritize Defects 381
Resources 381
Summary 382

Chapter 17: Iteration Kickoff      383
Iteration Planning 383
Testable Stories 393
Collaborate with Customers 396
High-Level Tests and Examples 397
Summary 403

Chapter 18: Coding and Testing      405
Driving Development 406
Tests that Critique the Product 412
Collaborate with Programmers 413
Talk to Customers 414
Completing Testing Tasks 415
Dealing with Bugs 416
It’s All about Choices 419
Facilitate Communication 429
Regression Tests 432
Resources 434
Iteration Metrics 435
Summary 440

Chapter 19: Wrap Up the Iteration       443
Iteration Demo 443
Retrospectives 444
Celebrate Successes 449
Summary 451

Chapter 20: Successful Delivery       453
What Makes a Product? 453
Planning Enough Time for Testing 455
The End Game 456
Customer Testing 464
Post-Development Testing Cycles 467
Deliverables 468
Releasing the Product 470
Customer Expectations 475
Summary 476

Part VI: Summary      479
Chapter 21: Key Success Factors       481
Success Factor 1: Use the Whole-Team Approach 482
Success Factor 2: Adopt an Agile Testing Mind-Set 482
Success Factor 3: Automate Regression Testing 484
Success Factor 4: Provide and Obtain Feedback 484
Success Factor 5: Build a Foundation of Core Practices 486
Success Factor 6: Collaborate with Customers 489
Success Factor 7: Look at the Big Picture 490
Summary 491

Glossary      493
Bibliography      501
Index      509