Research and Evaluation in Education and Psychology

Donna M. Mertens is Professor Emeritus, Department of Education, at Gallaudet University. She taught research methods and program evaluation to deaf and hearing students at the MA and PhD levels for over 30 years. She now teaches courses and professional development workshops around the world. She conducts research and evaluation studies on such topics as improvement of special education services in international settings, planning for the inclusion of students with disabilities in neighborhood schools, enhancing the educational experiences of students with disabilities, preventing sexual abuse in residential schools for deaf students, improving access to the court systems for deaf and hard-of-hearing people, and improving the preparation of teachers of the deaf through appropriate use of instructional technology. Her research focuses on improving methods of inquiry by integrating the perspectives of those who have experienced oppression in our society. She draws on the writings of feminists, racial and ethnic minorities, and people with disabilities, as well as Indigenous peoples who have addressed the issues of power and oppression and their implications for research methodology. Dr. Mertens has made numerous presentations at the meetings of the American Educational Research Association, American Evaluation Association, Australasian Evaluation Society, Association for College Educators of the Deaf and Hard of Hearing, International Sociological Association, Mixed Methods International Research Association, American Psychological Association, African Evaluation Association, Canadian Evaluation Society, Visitors Studies Association, and other organizations that explore these themes. She served as president and board member of the American Evaluation Association from 1997 to 2002 and as a member of the Board of Trustees for the International Organization for Cooperation in Evaluation, 2002–2003. She served as editor for the Journal of Mixed Methods Research 2009-2014. Her publications include four edited volumes, Indigenous Pathways to Social Research (coedited with Fiona Cram and Bagele Chilisa, 2013), Handbook of Social Research Ethics (coedited with Pauline Ginsberg, 2009), Creative Ideas for Teaching Evaluation (1989), and Research and Inequality (coedited with Carole Truman and Beth Humphries, 2000), and several authored books, including Mixed Methods Design in Evaluation (2018), Program Evaluation: A Comprehensive Guide (2nd ed.) (with Amy Wilson, 2018), Transformative Research and Evaluation (2009), Research and Evaluation Methods in Special Education (coauthored with John McLaughlin, 2004), and Parents and Their Deaf Children (coauthored with Kay Meadow-Orlans and Marilyn Sass Lehrer, 2003). She also publishes many chapters and articles in edited volumes, encyclopedias, handbooks, and journals, such as Journal of Mixed Methods Research, Qualitative Social Work, Eye on Psi Chi, Educational Researcher, International Journal of Mixed Methods Research, New Directions for Program Evaluation, American Journal of Evaluation, American Annals of the Deaf, Studies in Educational Evaluation, and Educational Evaluation and Policy Analysis.

Preface
Companion Website
Acknowledgments
About the Author
Chapter 1: An Introduction to Research and Ethical Practice
Why Bother?
What is Research?
Research Terminology
Approach Taken in This Book
Major Paradigms in Research: A Brief History of Research
Postpositivism
Constructivist Paradigm
Transformative Paradigm
Pragmatic Paradigm
Politics, Legislation, and the Paradigms
Contested Territory: Quality, Causality, and Objectivity
Merging Paradigms—Across Differences
Chapter 2: Evaluation
Defining Evaluation
Distinguishing Research and Evaluation
History and Models of Evaluation
Resources and Processes for Conducting Evaluations
Steps in Planning an Evaluation
Standards for Critically Evaluating Evaluations
Ethics and Evaluation: The Guiding Principles
Questions for Critically Analyzing Evaluation
Evaluation Resources for Program and Policy Evaluations
Chapter 3: Literature Review and Focusing the Research
Reasons for Doing Literature Reviews
The Search Process
Critical Analysis of Literature Reviews
Questions for Critically Analyzing Literature Reviews
Chapter 4: Experimental and Quasi-Experimental Research
Importance of Experimental Design
Research Designs and Threats to Validity
Experimental, Quasi-Experimental, and Single-Group Designs
Other Design Issues
Challenges to Using Experimental Designs in Educational and Psychological Research
Transformative Perspectives Regarding Experimental Research
Postpositivist Rejoinder
Final Thoughts
Questions for Critically Analyzing Experimental, Quasi-Experimental, and Single Group Designs
Where to From Here?
Chapter 5: Causal Comparative and Correlational Approaches
Inherent Characteristics: Challenging Issues
Causal Comparative Research
Correlational Research
Questions for Critically Analyzing Causal Comparative and Correlational Research
Chapter 6: Survey Methods
Design Phase
Sampling Plan
Designing the Survey Instrument
Pilot Testing the Questionnaire
Language Differences between the Researcher and the Respondent
Special Types of Questions
Letter of Transmittal
Conducting the Survey
Data Analysis and Troubleshooting Your Survey Instrument
Questions for Critically Analyzing Survey Research
Chapter 7: Single-Case Research
Quality Issues in Single-Case Research
Design Options
Data Analysis In Single-Case Research
Questions for Critically Analyzing Single-Case Research
Chapter 8: Qualitative Research and Action Research
Basis for Selection of Qualitative Methods
Types of Qualitative Research
Data Collection
General Methodological Guidelines
Translation Issues in Multilingual Contexts
Qualitative Data Collection Methods
Critically Analyzing Qualitative Research
Questions for Critically Analyzing Qualitative Research
Chapter 9: History, Narrative Study of Lives, and Autoethnography
Why Study History?
Importance of Historical, Narrative, and Autoethnographic Research
Types of Historical, Narrative, and Autoethnographic Research
Steps in Conducting Historical-Narrative Research
Notes on Biographical Research
Notes on Autoethnographic, Autobiographical/Self-Study Research
Questions for Critically Analyzing Historical-Narrative Research
Chapter 10: Mixed Methods Research
Definition and Characteristics
Importance in Educational and Psychological Research
Philosophical Assumptions and Mixed Methods Research
Mixed Methods Design Options
Questions for Critically Analyzing Mixed Methods Research
Chapter 11: Sampling
Definition, Selection, and Ethics
External Validity (Generalizability) or Transferability
Defining the Population and Sample
Identification of Sample Members
Sampling Strategies
Sampling Bias
Access Issues
Sample Size
Ethics and Protection of Study Participants
Questions for Critically Analyzing Sampling Strategies
Chapter 12: Data Collection
Operationalizing Concepts
Quantitative Measurement Topics
Selection and Development of Quantitative Instruments
Developing a Data Collection Instrument
Qualitative Data Collection Methods
Mixed Methods and Data Collection
Standards for Judging Quality of Data Collection
Questions for Critically Analyzing Data Collection
Chapter 13: Data Analysis, Interpretation, and Use
Quantitative Analysis Strategies
Qualitative Analytic Strategies
Analytic and Interpretive Issues in Mixed Methods
A Research Plan: The Management Plan and Budget
Writing Research Reports
Utilization of the Research Results
Questions for Critically Analyzing Data Analysis and Interpretation