Social Media, Sociality, and Survey Research

Craig A. Hill, PhD, is Senior Vice President for the Survey, Computing, and Statistical Sciences at RTI International. He has more than thirty years of experience in survey research, having directed survey research projects for a wide variety of federal, academic, and commercial clients. Elizabeth Dean, MA, is a Survey Methodologist at RTI International. She specializes in the development and testing of innovative applications of survey methodology, such as designing surveys for various social media platforms, investigating the use of virtual worlds to increase survey privacy, and adapting cognitive pretesting methods for use with emerging technologies. Joe Murphy, MA, is a Survey Methodologist at RTI International. His research focus includes the implementation of new data collection processes and analytic techniques to maximize data quality, increase response, and reduce costs, as well as the role of new technologies and social media in the collection and analysis of social data.

List of Figures xiii

List of Tables xvii

Contributors xix

Preface xxi

Acknowledgments xxv

1. Social Media, Sociality, and Survey Research 1
Joe Murphy, Craig A. Hill, and Elizabeth Dean

What Is Social Media? 2

Social Media Origins 6

Social Networking Sites and Platforms 6

Blogs 8

Twitter 8

Facebook 9

LinkedIn 9

Second Life 9

Other Social Networking Platforms and Functionalities 10

Why Should Survey Researchers Be Interested in Social Media? 11

The Current State of Survey Research 11

Falling Response Rates 11

Frame Coverage Errors 13

The Coming Age of Ubiquity 14

Public vs. Private Data 17

Social Media Interaction: Next Wave (or Subwave)? 18

Adding Social Media to the Survey Research Toolbox 21

Toward Using the Concept of Sociality in Survey Research of the Future 22

How Can Survey Researchers Use Social Media Data? 26

References 28

2. Sentiment Analysis: Providing Categorical Insight into Unstructured Textual Data 35
Carol Haney

Describing Emotional or Subjective Feeling in Textual Data 36

Definition of Machine-Augmented Sentiment Analysis 37

How Sentiment Analysis Is Used with Text Data 38

Different Ways of Representing Sentiment 42

Ordinal Scales 42

Nominal Emotion Classification 43

Neutral Sentiment 44

Techniques for Determining Sentiment 44

Precursors to Analysis 44

Harvesting 46

Structure and Understand 50

Approaches to Determining Sentiment 51

Machine-Coded Sentiment Analysis 51

Human-Coded Sentiment Analysis 53

Sentiment Analysis as a Subset of Text Analytics 54

Current Limitations of Sentiment Analysis 57

References 59

3. Can Tweets Replace Polls? A U.S. Health-Care Reform Case Study 61
Annice Kim, Joe Murphy, Ashley Richards, Heather Hansen, Rebecca Powell, and Carol Haney

Methods 64

Twitter Data 64

Public Opinion About Health-Care Reform: Kaiser Health Tracking Poll 70

Analysis 70

Results 71

RQ1: To What Extent Was Health-Care Reform Discussed on Twitter? 71

RQ2: What Is the Distribution of Sentiment of Health-Care Reform Tweets? 74

RQ3. Do Trends in the Sentiment of Tweets About Health-Care Reform Correlate with Observed Trends
in Public Opinion About Health-Care Reform from Nationally Representative Probability-Based Surveys? 75

KFF Trends 75

Comparison 77

RQ4. What Are the Key Topics Discussed in Health-Care Reform Tweets? 78

Discussion 80

Conclusions 84

References 85

4. The Facebook Platform and the Future of Social Research 87
Adam Sage

The Changing Web: From Searchable to Social 88

Digital and Digitized Data 93

The Case for Facebook Integration 94

Data and the Graph API 97

Facebook Applications 99

Social Plugins 103

The Future, Mobile Apps, and the Ever Increasing Complexity of the Social Graph 104

References 104

5. Virtual Cognitive Interviewing Using Skype and Second Life 107
Elizabeth Dean, Brian Head, and Jodi Swicegood

Brief Background on Cognitive Interviews 108

Cognitive Interviewing Current Practice 109

Practitioners’ Techniques 109

Cognitive Interviews in Practice: Present and Future 112

Second Life for Survey Research 114

Methods 115

Recruitment 115

Screening 117

Incentive 118

Think-Aloud and Probes 118

Results 118

Overall Participant Characteristics 118

Feasibility of Pilot Study 120

Quality of Cognitive Interviews by Mode 121

Participant Disengagement 122

Nonverbal Cues 125

Total Problems 126

Type and Severity of Problems 126

Conclusions 127

Discussion and Future Research 128

References 129

6. Second Life as a Survey Lab: Exploring the Randomized Response Technique in a Virtual Setting 133
Ashley Richards and Elizabeth Dean

Overview of Second Life 134

Research in Second Life 134

The Randomized Response Technique 136

Study Design 137

Results 142

Discussion 144

References 146

7. Decisions, Observations, and Considerations for Developing a Mobile Survey App and Panel 149
David Roe, Yuying Zhang, and Michael Keating

Impact of the Evolution of Technology on Data Collection 150

Telephone Interviewing 151

Web Interviewing 151

Cell Phones 152

Smartphones 153

Building an App 156

Goals 157

Preliminary Findings 168

Recruitment 170

Respondent Communication 170

Survey Topics 172

Respondent Impressions on Incentives, Survey Length, and Frequency 175

Next Steps 175

References 176

8. Crowdsourcing: A Flexible Method for Innovation, Data Collection, and Analysis in Social Science Research 179
Michael Keating, Bryan Rhodes, and Ashley Richards

What Is Crowdsourcing? 180

Open Innovation 181

Cisco Systems I-Prize Challenge 182

RTI International’s 2012 Research Challenge 183

Options for Hosting Your Own Challenges 185

Legal Considerations 186

Data Collection 187

Crowdsourcing Survey Response on Mechanical Turk 187

Targeted Data Collection 190

Cost Considerations 194

MyHeartMap Challenge 195

Analysis by Crowdsourcing 197

Sentiment Analysis 197

Challenge-Based Data Analysis 198

Conclusion 199

References 200

9. Collecting Diary Data on Twitter 203
Ashley Richards, Elizabeth Dean, and Sarah Cook

Background 204

Twitter 204

Diaries 204

Methods 206

Recruitment 208

Data Collection 210

Results 211

Nonresponse 212

Data Quality 216

Incentive Preference 221

Participant Feedback 222

Discussion 227

References 229

10. Recruiting Participants with Chronic Conditions in Second Life 231
Saira N. Haque and Jodi Swicegood

Background 233

Methods 234

Instrument Development 235

Recruitment Methods 235

Survey Administration 244

Results 244

Discussion 247

Communities 247

Using Existing Second Life Resources 248

Other Effective Methods 249

The Importance of the Recruitment Avatar 249

Conclusion 250

References 251

11. Gamification of Market Research 253
Jon Puleston

Significance of Gamification in Market Research 254

Apply Gamification to Market Research 256

Gamification in Survey Design 259

Apply Rules to Question Design 265

Add the Competitive Element 269

Add Reward Mechanics 271

Give Feedback 272

Make Tasks More Involving 273

Ensure the Challenge Can Be Accomplished 275

How to Design Questions To Be More Game-Like 275

Common Questions About Gamification 284

Who Responds to Gamification? 284

What Impact Does Gamification Have on the Data? 285

How Do These Techniques Work in Different Cultures? 289

Conclusions 291

References 292

12. The Future of Social Media, Sociality, and Survey Research 295
Craig A. Hill and Jill Dever

Statistical Challenges with Social Media Data 296

Quality and Representativeness 297

Sampling from Social Media Sources 298

Population Estimation from Social Media Data 303

Future Opportunities 306

What Does the Future Hold? 307

Sociality Hierarchy Level 1: Broadcast 308

Sociality Hierarchy Level 2: Conversation 311

Sociality Hierarchy Level 3: Community 312

Final Thoughts 314

References 315

Index 319