Questionnaires in Second Language Research

Zoltán Dörnyei was a Hungarian-born British applied linguist. He was Professor of Psycholinguistics at the University of Nottingham, UK. He was one of the most prolific and influential researchers in second language acquisition and the psychology of foreign language learning. His work on motivation remains central in the field. He was awarded the Kenneth W. Mildenberger Prize in 1989. Jean-Marc Dewaele is Professor of Applied Linguistics and Multilingualism at Birkbeck, University of London, UK. He has published widely on individual differences. He is former President of the European Second Language Association (EUROSLA) and is General Editor of the Journal of Multilingual and Multicultural Development. He won the Robert Gardner Award for Excellence in Second Language and Bilingualism Research in 2016 and the EUROSLA Distinguished Scholarship Award in 2022.

Table of Contents

Preface to the Third Edition

Introduction

1 Questionnaires in Second Language Research

1.1 What Are "Questionnaires" and What Do They Measure?

1.1.1 What a Questionnaire Is Not

1.1.2 What Do Questionnaires Measure?

1.2 Using Questionnaires: Pros and Cons

1.2.1 Advantages

1.2.2 Disadvantages

1.3 Questionnaires in Quantitative and Qualitative

2 Constructing the Questionnaire

2.1 General Features

2.1.1 Length

2.1.2 Layout

2.1.3 Sensitive Topics and Anonymity

2.2 The Main Parts of a Questionnaire

2.2.1 Title

2.2.2 Instructions

2.2.3 Questionnaire Items

2.2.4 Sociobiographical Information

2.2.5 Additional Information

2.2.6 Final "Thank You"

2.3 Appropriate Sampling of the Questionnaire Content and the Significance of "Multi-Item Scales"

2.3.1 Appropriate Sampling of the Content

2.3.2 Using Multi-Item Scales

2.4 "Closed-Ended" Questionnaire Items

2.4.1 Rating Scales

2.4.2 Multiple-Choice Items

2.4.3 Rank Order Items

2.4.4 Numeric Items

2.4.5 Checklists

2.5 Open-Ended Questions

2.5.1 Specific Open Questions

2.5.2 Clarification Questions

2.5.3 Sentence Completion Items

2.5.4 Short-Answer Questions

2.6 How to Write Good Items

2.6.1 Drawing Up an "Item Pool"

2.6.2 Rules About Writing Items

2.6.3 Writing Sensitive Items

2.7 Grouping and Sequencing Items

2.7.1 Clear and Orderly Structure

2.7.2 Opening Questions

2.7.3 Factual (or "Personal" or "Classification") Questions at the Beginning or that the End?

2.7.4 Open-Ended Questions at the End

2.8 Translating the Questionnaire

2.8.1 Translation as a Team-Based Approach

2.8.2 Translation with Limited Resources

2.9 Computer Programs for Constructing Questionnaires

2.10 Piloting the Questionnaire and Conducting Item Analysis

2.10.1 Initial Piloting of the Item Pool

2.10.2 Final Piloting ("Dress Rehearsal")

2.10.3 Item Analysis

3 Administering the Questionnaire

3.1 Selecting the Sample

3.1.1 Sampling Procedures

3.1.2 How Large Should the Sample Be?

3.1.3 The Problem of Respondent Self-Selection

3.2 Main Types of Questionnaire Administration

3.2.1 Administration by Mail

3.2.2 One-to-One Administration

3.2.3 Group Administration

3.3 Strategies to Increase the Quality and Quantity of Participant Response

3.3.1 Advance notice

3.3.2 Attitudes Conveyed by Teachers, Parents, and Other Authority Figures

3.3.3 Respectable Sponsorship

3.3.4 The Presence of a Survey Administrator

3.3.5 The Behavior of the Survey Administrator

3.3.6 Communicating the Purpose and Significance of the Survey

3.3.7 Emphasizing Confidentiality

3.3.8 Reading Out the Questionnaire Instructions

3.3.9 The Style and Layout of the Questionnaire

3.3.10 Promising Feedback on the Results

3.4 Questionnaire Administration, Confidentiality, and Other Ethical Issues

3.4.1 Basic Ethical Principles of Data Collection

3.4.2 Obtaining Consent for Children

3.4.3 Strategies for Getting Around Anonymity

4 Processing Questionnaire Data

4.1 Coding Questionnaire Data

4.1.1 First Things First: Assigning Identification Codes

4.1.2 Coding Quantitative Data

4.2 Inputting the Data

4.2.1 Creating and Naming the Data File

4.2.2 Keying in the Data

4.3 Processing Closed Questions

4.3.1 Data Cleaning

4.3.2 Data Manipulation

4.3.3 Reducing the Number of Variables in the Questionnaire

4.3.4 Main Types of Questionnaire Data

4.3.5 Examining the Reliability and Validity of the Data

4.3.6 Statistical Procedures to Analyze Data

4.4 Content Analysis of Open-Ended Questions

4.5 Computer Programs for Processing Questionnaire Data

4.6 Summarizing and Reporting Questionnaire Data

4.6.1 General Guidelines

4.6.2 Technical Information to Accompany Survey Results

4.6.3 Reader-Friendly Data Presentation Methods

4.7 Complementing Questionnaire Data with Other Information

4.7.1 Questionnaire Survey with Follow-up Interview or Retrospection

4.7.2 Questionnaire Survey Facilitated by Preceding Interview

5 Online questionnaires

5.1 The effect of setting and social obligation in paper and online questionnaires

5.1.1 Social context in online questionnaires

5.1.2 The ephemeral relationship between participant and researcher

5.2 Fear and distrust of all types of questionnaires

5.3 When online questionnaires are impractical or impossible

5.4 Non-probability sampling in online questionnaires

5.5 Developing a good online questionnaire

5.5.1 Knowing something about statistics before creating a questionnaire

5.5.2 Choosing a platform

5.5.3 Pilot testing the online questionnaire

5.6 Getting the snowball to roll

5.6.1 Producing an attractive call for participation

5.6.2 The appearance of the questionnaire

5.6.3 The use of a progression bar

5.6.4 Friendly academic nudging

5.6.5 Rewarding participants

5.7 When the harvest is in

5.7.1 Cleaning the data

5.7.2 Calculating internal consistency

5.7.3 Being careful with automatic coding

5.8 Running the statistics to answer the research questions: the Hallelujah moment

5.8.1 Double-checking suspicious results

5.8.2 Playing by the rules

5.8.3 General Data Protection Regulation (GDPR)

6. Developing psychometrically sound instruments

6.1 Developing and Validating the Short-form Foreign Language Enjoyment Scale (S-FLES) (Botes et al., 2021)

6.1.1 The background

6.1.2 Developing a Short-Form through Sequential Steps

6.1.3 Concluding Remarks on the S-FLES

6.1.4 Final Short-form Foreign Language Enjoyment Scale (S-FLES)

6.2 Developing and validating a questionnaire to measure intensity and quality of L2 engagement (Teravainen-Goff, in preparation)

6.2.1 Background

6.2.2 Key challenges in developing the intensity and quality of L2 engagement questionnaire

6.2.3 Final version of intensity and quality of L2 engagement questionnaire (Teravainen-Goff, in preparation)

6.3 Developing and validating the Foreign Language Learning Boredom Scale (FLLBS) (Li et al., 2020)

6.3.1 Initial conceptualization of the target construct

6.3.2 Development of the initial questionnaire

6.3.3 Validation of the questionnaire

6.3.4 Final Foreign Language Learning Boredom Scale and its English translation

7. Conclusion and Checklist

References

Appendix

Author Index

Subject Index