Conducting Online Research on Amazon Mechanical Turk and Beyond (eBook)

Chapter 1: Introduction
A Scientific Revolution in the Making
A Brief History of Online Research in the Social and Behavioral Sciences: From HTML 2.0 to Mechanical Turk
The Use of Online Samples in Applied Behavioral Research
Amazon Mechanical Turk
Chapter 2: The Mechanical Turk Ecosystem - Leib Litman, Cheskie Rosenzweig, Jonathan Robinson
How Quality is Maintained
Reputation Mechanism
Selectively Recruiting Specific Workers
Protections for Workers
Communicating with Workers
A Worker’s Perspective
Worker Communities
Chapter 3: Conducting a Study on Mechanical Turk
Sample Project
Setting up a Requester Account on Mechanical Turk
Creating a HIT
The ‘Design Layout’ tab
Monitoring Progress on the Requester’s Dashboard
When a Worker Runs Out of Time
Sample Study Results
Conducting Follow-up Studies Using Requester-Issued Qualifications
Appendix A: Checklist for best practices of setting up a Mechanical Turk HIT
Chapter 4: API and Third Party Apps
Third Party API-based Platforms
Common Uses for API Scripts and Third Party API-based Apps
TurkPrime
Chapter 5: Data Quality Issues on MTurk - Jesse Chandler, Gabriele Paolacci, David Hauser
Defining and Measuring Data Quality
Measuring Individual Participant Data Quality
Causes of and Cures for Poor Data Quality
Concluding Thoughts
Chapter 6: Who are the Mechanical Turk Workers?
Sources of Data
Location of Workers in the US
Demographics of Mechanical Turk
Chapter 7: Sampling Mechanical Turk Workers: Problems and Solutions
Sampling on Mechanical Turk
Sources of Sampling Bias
The Problem of Superworkers
Time-of-day Effects
Pay Rate
Dropout
Sampling Best Practices
Chapter 8: Data Representativeness of Mechanical Turk Samples
Representativeness, surveys, and survey sampling
The methodology of survey sampling
Mechanical Turk as a sampling frame
The fit-for-purpose framework
Chapter Overview:
Chapter 9: Conducting Longitudinal Research on Amazon Mechanical Turk
Why Longitudinal Research?
Retention, Longitudinal Research, and MTurk
Case Studies
Best practices for longitudinal research
Chapter 10: Beyond Mechanical Turk: Using Online Market Research Platforms
Limitations of MTurk
Online probability-based panels
Online market research platforms
Overall comparisons between Mechanical Turk and market research platforms
Chapter 11: Conducting Ethical Online Research: A Data-Driven Approach
Historical background
Risk of harm in online research
Research on sensitive topics
A deeper dive into controversial and complex issues
Economics of Mechanical Turk: Considerations for setting wages
Setting wages: Considerations of ethics and methodology
Considerations for rejecting, blocking and disqualifying workers.
Practical advice for requester/worker interactions
Anonymity
Appendix A