Usability Testing of Medical Devices

Wiklund Research & Design, Inc., Concord, Massachusetts, USA Wiklund Research Design, Concord, MA

Introduction
What is usability testing?
What is a medical device?
Why conduct usability tests of medical devices?
Does usability matter to regulators?
Is usability testing of medical devices required?
Do you have to test minor design changes?
How do you defend usability testing methods to market researchers?

Risk Management and Usability Testing
What is the relationship between usability testing and risk management?
Can usability testing identify use-related hazards?
What is a dangerous use error?
Is usability testing a reliable way to assess the likelihood that a dangerous use error will occur?

The Commercial Imperative
Can you develop marketing claims based on test results?
How does testing affect the development schedule?
Does usability testing offer liability protection?

Testing Costs
What should a request for quotation for usability testing include?
What does a usability test cost?
What is the return on investment?

Anatomy of a Usability Test
What are the common elements of a usability test?
What is a test session’s proper duration?
Do you have to be a usability specialist to conduct a test?
Does it take a "brain surgeon" to evaluate medical devices?
Why test if you cannot change the design?
How do you set expectations?
What can postpone a usability test?

Types of Tests
What is the difference between formative and summative usability testing?
What is a benchmark usability test?
What is an "out of the box" usability test?
Can a test session include more than one participant?
Can you conduct a group test?
How do you conduct a "quick and dirty" usability test?

Writing a Test Plan
What should a test plan include?
What are the key elements of a usability test plan from a regulatory perspective?
Do usability test plans require Institutional Review Board (IRB) approval?
How do you protect intellectual property

Choosing a Participant Sample and Recruiting Participants
What is an appropriate sample size?
Can advisory panel members play a role in usability tests?
Should children participate in usability tests?
Should seniors participate in usability tests?
How do you conduct a usability test involving people with impairments?
How do you recruit test participants?
How do you recruit physicians?
How do you recruit nurses?
How do you recruit laypersons?
How do you prevent "no-shows?"

Test Environments
What is the benefit of testing in a medical environment simulator?
How do you test in actual use environments?
Should you test in a participant’s workplace?
Can you conduct a usability test over the Web?
Can you test a device while it’s in actual use?
What if a "device" cannot be moved?

Adding Realism
Why and how do you distract test participants?
What role can a "standardized patient" play?
What use is a mannequin?
How do you simulate invasive procedures?
How do you simulate blood?
How do you simulate skin and injections?
How do you simulate impairments?
How do you simulate hardware interactions?
How do you simulate other medical devices?

Selecting Tasks
Do you have to test everything?
What tasks should test participants perform?
Why focus on potentially dangerous tasks?
How do you choose tasks when evaluating use-safety?
Should tests include maintenance and service tasks?
Can you test long-term usability?
How do you test alarms?
How do you test warning labels?
How do you test instructions for use?
How do you test symbols?
How do you test legibility?
How do you evaluate packaging?
How do you test a device’s appeal?

Conducting the Test
What’s the value of pilot testing?
Who should observe the test sessions?
What kinds of usability problems arise during a usability test?
What can go wrong before, during, and after a test?
What risk do test personnel assume?
Are there times when the testing staff should be all-female or all-male?
Should user interface designers conduct usability tests of their own designs?
When and how should you assist test participants?
Can you modify a test-in-progress?
Can you reliably detect use errors?
Can you give test participants training?
Should you provide access to learning tools?

Interacting with Test Participants
When is it appropriate to ask participants to think aloud?
What’s the proper way to pose a question?
Is there a place for humor in a usability test?
How do you minimize participant fatigue?
How do you protect participants from harm?
What if the test participant gets hurt?

Documenting the Test
What data should you collect?
What use are task times?
What’s a good way to video record a session?
How do you video record participants’ interactions with a moving device?

Analyzing Test Data
What kind of statistical analyses are most useful?
How do you handle outliers?
How do you deal with use errors that are not design-related?

Reporting Results
What makes a good test report?
Should test reports include design recommendations?
Can usability test results be misleading?
How do you deliver bad news?
How do you explain a lack of statistical significance?
What goes into making a highlight video?

Validation Testing
How does design validation differ from design verification?
Do you need to conduct a test prior to filing for an Investigational Device Exemption?
Can a clinical trial supplant summative usability testing?
Can you conduct a usability test in parallel with a clinical trial?
Can you conduct a summative usability test without conducting a formative usability test?
Resources
Index