Survey Measurements

Uwe Engel ist Professor für Soziologie mit dem Schwerpunkt Statistik und empirische Sozialforschung an der Universität Bremen.

Contents

Preface7
Uwe Engel

1. Introduction9

Uwe Engel

2. Motivated Misreporting: Shaping Answers to Reduce Survey Burden24

Roger Tourangeau, Frauke Kreuter, and Stephanie Eckman

3. Audio-recording of Open-ended Survey Questions: A Solution to the Problems of Interviewer Transcription?42

Patrick Sturgis and Rebekah Luff

4. Framing Effects58

Uwe Engel and Britta Köster

5. Estimating and Comparing the Quality of Different Scales of an Online Survey Using an MTMM Approach76

Melanie Revilla and Willem E. Saris

6. Collecting MTMM Data on Satisfaction with Life97

Laura Burmeister and Uwe Engel

7. On the Quality of Web Panels112

Jelke Bethlehem

8. Online Surveys and the Burden of Mobile Responding130

Marika de Bruijne and Marije Oudejans

9. Well-being, Survey Attitudes, and Readiness to Report on Everyday Life Events in an Experience Sampling Study146

Laura Burmeister, Uwe Engel, and Björn Oliver Schmidt

10. Nonresponse, Measurement Error, and Estimates of Change - Lessons from the German PPSM Panel160

Suat Can and Uwe Engel

11. Handling of Missing Data in Statistical Analyses192

Daniel Salfrán and Martin Spiess

12. Multiple Imputation of Overdispersed Multilevel Count Data209

Kristian Kleinke and Jost Reinecke

Contributors227

Subject Index229

Author Index235

1. Introduction

Uwe Engel

1.1 Data Quality

Surveys are important for society. They are frequently conducted and useful sources of public opinion and decision making. Although even out-comes of high-quality surveys are not safe from being misinterpreted, ei-ther inadvertently or even deliberately, high-quality survey data are likely to reduce this risk. For scientific reasons as well, strictly speaking only high-quality survey data appear acceptable. This is why survey methodology pays so much attention to possible threats to data quality and has been doing so for quite some time (e.g. Biemer and Lyberg 2003, Weisberg 2005).

Why is high data quality so important for survey research? One possible answer to this question may point to the risk of obtaining biased sample estimates of population parameters if a survey fails to cope with relevant sources of survey error. Probability sampling and proper use of statistical estimators alone cannot guarantee unbiased estimates, because even in this case nonresponse and measurement effects may still give rise to bias and error variance.

Accordingly, one core task certainly consists in the development of suitable statistical models and techniques to adjust for nonresponse bias. Even the ideal case of complete (or perfectly nonresponse-adjusted for) response, however, cannot guarantee unbiased samples estimates for a simple reason: Observed responses may deviate from their corresponding true scores due to measurement effects.

Such effects may have different origins, including the survey mode, question wordings, and response formats. In addition to such 'mode' and 'response effects', the 'interviewer' represents a further source of meas-urement error. Of importance is also the 'respondent' insofar as his/her response behavior may differ in relevant aspects. In this respect, typical examples are certainly satisficing behavior and cognitive response styles. Another source of variation is simply that respondents arrive at their re-sponses to survey questions through cognitive processes that may differ in relevant regards. No less important than this, however, is another factor of answering behavior which might be called 'motivated misreporting'.

A working definition of 'high-quality' surveys might thus include the idea that the quality is the higher the more such sources of survey error are effectively controlled for. In doing this, one would adopt the prominent 'total survey error' perspective.

1.2 Sources of Survey Error

1.2.1 Measurement Error

Measurement error may be due to several sources of variation that affect response behavior. Surveys do not yield unobtrusive measurements. In-stead, already the fact per se that respondents are asked questions in the context of research interviews shapes their answering behavior in some ways.

Survey-mode effects

It is well known that different survey modes produce different mean val-ues, other things being equal. It makes a difference whether a finding has been obtained in an interviewer-assisted or self-administered survey mode. For instance, the analysis presented in chapter 10 below exemplifies the typical observation that the web mode tends to produce lower mean values than the telephone mode. 'Lower' means at the same time 'farther away' from an answer the respondent is assumed to believe to be an expected, i.e. socially desirable, answer. In the aforesaid analysis, this is the assumed expectation of presenting oneself as currently satisfied with one's life. If posed in direct communication, the mere presence of an interviewer gives rise to a kind of 'positivity bias' (Tourangeau et al. 2000, 240f.) and this bias in turn to a comparatively higher mean value than observed in the opposite case of self-administered survey modes. This is just an example of a kind of measurement effect which is usually termed 'mode effect'.

Response effects

Other measurement effects are called res