Measuring the User Experience

William (Bill) Albert is Senior Vice President and Global Head of Customer Development at Mach49, a growth incubator for global businesses. Prior to joining Mach49, Bill was Executive Director of the Bentley University User Experience Center (UXC) for almost 13 years. Also, he was Director of User Experience at Fidelity Investments, Senior User Interface Researcher at Lycos, and Post-Doctoral Researcher at Nissan Cambridge Basic Research. He has more than twenty years of experience in user experience research, design, and strategy. Bill has published and presented his research at more than 50 national and international conferences, and published in many peer-reviewed academic journals within the fields of User Experience, Usability, and Human-Computer Interaction. In 2010 he co-authored (with Tom Tullis and Donna Tedesco), “Beyond the Usability Lab: Conducting Large-Scale Online User Experience Studies,” published by Elsevier/Morgan Kauffman. Thomas S. (Tom) Tullis retired as Vice President of User Experience Research at Fidelity Investments in 2017. Tom was also an Adjunct Professor in Human Factors in Information Design at Bentley University since 2004. He joined Fidelity in 1993 and was instrumental in the development of the company’s User Research department, whose facilities include state-of-the-art Usability Labs. Prior to joining Fidelity, he held positions at Canon Information Systems, McDonnell Douglas, Unisys Corporation, and Bell Laboratories. He and Fidelity’s usability team have been featured in a number of publications, including Newsweek, Business 2.0, Money, The Boston Globe, The Wall Street Journal, and The New York Times.

Introduction; Background: Data Types; Sampling Size; Experimental Design; Data Analysis. Overview of Usability Metrics: Types of Metrics; Methods and Metrics; Summative vs. Formative; Choosing Appropriate Metrics. Performance Metrics: Task Success; Completion Time; Errors; Efficiency (clicks, pages, steps, etc.). Issues-Based Metrics: What is a Usability Issue; Severity Ratings; Test Biases; Reporting Positive Issues. Peferential-Based Metrics: Satisfaction; Ease of Use, Usefulness; Expectations; Standard Questionnaires. Web Navigation Metrics: Web-page Click-through Rates; Web page Abandonment Rates. Derived Metrics: Task-based; Aggregate. Observational Metrics: Eye Movements; Stress; Facial Expressions; Other Observational Metrics. Case Studies. Special Topics: Six Sigma and Usability; Automated Methods; Discount Techniques; Server Log Analysis; A/B Testing. Conclusion: Communication to Management; Cost Justification; Industry Trends.