Research and Evaluation in Education and Psychology

Donna M. Mertens is Professor Emeritus, Department of Education, at Gallaudet University. She taught research methods and program evaluation to deaf and hearing students at the MA and PhD levels for over 30 years. She now teaches courses and professional development workshops around the world. She conducts research and evaluation studies on such topics as improvement of special education services in international settings, planning for the inclusion of students with disabilities in neighborhood schools, enhancing the educational experiences of students with disabilities, preventing sexual abuse in residential schools for deaf students, improving access to the court systems for deaf and hard-of-hearing people, and improving the preparation of teachers of the deaf through appropriate use of instructional technology. Her research focuses on improving methods of inquiry by integrating the perspectives of those who have experienced oppression in our society. She draws on the writings of feminists, racial and ethnic minorities, and people with disabilities, as well as Indigenous peoples who have addressed the issues of power and oppression and their implications for research methodology. Dr. Mertens has made numerous presentations at the meetings of the American Educational Research Association, American Evaluation Association, Australasian Evaluation Society, Association for College Educators of the Deaf and Hard of Hearing, International Sociological Association, Mixed Methods International Research Association, American Psychological Association, African Evaluation Association, Canadian Evaluation Society, Visitors Studies Association, and other organizations that explore these themes. She served as president and board member of the American Evaluation Association from 1997 to 2002 and as a member of the Board of Trustees for the International Organization for Cooperation in Evaluation, 2002–2003. She served as editor for the Journal of Mixed Methods Research 2009-2014. Her publications include four edited volumes, Indigenous Pathways to Social Research (coedited with Fiona Cram and Bagele Chilisa, 2013), Handbook of Social Research Ethics (coedited with Pauline Ginsberg, 2009), Creative Ideas for Teaching Evaluation (1989), and Research and Inequality (coedited with Carole Truman and Beth Humphries, 2000), and several authored books, including Mixed Methods Design in Evaluation (2018), Program Evaluation: A Comprehensive Guide (2nd ed.) (with Amy Wilson, 2018), Transformative Research and Evaluation (2009), Research and Evaluation Methods in Special Education (coauthored with John McLaughlin, 2004), and Parents and Their Deaf Children (coauthored with Kay Meadow-Orlans and Marilyn Sass Lehrer, 2003). She also publishes many chapters and articles in edited volumes, encyclopedias, handbooks, and journals, such as Journal of Mixed Methods Research, Qualitative Social Work, Eye on Psi Chi, Educational Researcher, International Journal of Mixed Methods Research, New Directions for Program Evaluation, American Journal of Evaluation, American Annals of the Deaf, Studies in Educational Evaluation, and Educational Evaluation and Policy Analysis.

Preface
Acknowledgments
Contents
A Eulogy to Diversity
An Introduction to Research
Why Bother?
What Is Research?
Research Terminology
Approach Taken in This Book
Major Paradigms in Research: A Brief History of Research
The Politics of Acceptance: The Objectivity Debate
Politics, Legislation and the Paradigms
Ethics in Research
Merging Paradigms
Questions and Activities for Discussion and Application
Evaluation
Defining Evaluation
History and Current Models of Evaluation
Steps in Conducting Evaluations
Standards for Critically Evaluating Evaluations
Questions for Critically Analyzing Evaluation Studies
Questions and Activities for Discussion and Application
Literature Review and Research Problems
Major Reasons for Doing Literature Reviews
The Search Process
Questions for Critically Analyzing Literature Reviews
Questions and Activities for Discussion and Application
Experimental and Quasi-Experimental Research
Single-Group, Experimental, and Quasi-Experimental Designs
Research Designs
Challenges to Using Experimental Designs in Educational and Psychological Research
Questions for Critically Analyzing Single-Group, Experimental, and Quasi-Experimental Designs
Questions and Activities for Discussion and Application
Causal Comparative and Correlational Research
Types of Variables
Causal Comparative Research
Correlational Research
Questions for Critically Analyzing Causal Comparative and Correlational Research
Questions and Activities for Discussion and Application
Survey Research
Design Phase
Data Collection Choices
Feminist Perspectives
Disability Perspectives
Sampling Plan
Designing the Questionnaire
Formatting the Questionnaire
Pilot Test the Questionnaire
Language Differences Between the Researcher and the Respondent
Special Types of Questions
Letter of Transmittal
Conducting the Survey
Response Rate Considerations
Notes Specific to Phone Interviews
Notes Specific to Personal Interviews
Transformative Perspectives on Interviewing
Notes Specific to Web Based Surveys
Data Analysis With Survey Research
Questions for Critically Analyzing Survey Research
Questions and Activities for Discussion and Application
Single-Case Research
Quality Issues With Single-Case Research
Design Options
Data Analysis in Single-Case Research
Questions for Critically Analyzing Single-Case Research
Questions and Activities for Discussion and Application
Qualitative Methods
Basis for Selection of Qualitative Methods
Strategies for Qualitative Research
General Methodological Guidelines
Critically Analyzing Qualitative Research
Questions for Critically Analyzing Qualitative Research
Questions and Activities for Discussion and Application
History and Narrative Study of Lives
Importance of Historical and Narrative Research
Types of Historical and Narrative Research
Steps in Conducting Historical-Narrative Research
Special Notes on Biographical Research
Special Notes on Autobiographical/Self-Study Research
Questions for Critically Analyzing Historical-Narrative Research
Questions and Activities for Discussion and Application
Mixed methods and mixed model designs
Definition and characteristics
Importance in educational and psychological research
Philosophical assumptions: Pragmatism, mixing paradigms, and transformation
Mixed methods/models design options
Critically analyzing mixed methods research
Questions and activities for discussion and application
Sampling: Definition, Selection, and Ethics
Sampling Strategies: Alternative Paradigms
External Validity (Generalizability) or Transferability
Defining the Population and Sample
Identification of Sample Members
Sampling Strategies
Sampling Bias
Access Issues
Access to Records
Sample Size
Ethics and Protection of Study Participants
Questions for Critically Analyzing Sampling Strategies
Questions and Activities for Discussion and Application
Data Collection
Operationalizing Concepts
Standards for Judging Quality of Data Collection
Quantitative Measurement Topics
Selection and Development of Quantitative Instruments
Developing a Data Collection Instrument
Qualitative Data Collection Methods
Questions for Critically Analyzing Data Collection
Activities and Questions for Discussion and Application
Data Analysis, Interpretation, and Reporting
Quantitative Analysis Strategies
Special Application: Statistical Synthesis as Literature Review
Qualitative Analytic Strategies
A Research Plan: The Management Plan and Budget
Writing Research Reports
Utilization of the Research Results
Questions for Critically Analyzing Data Analysis and Interpretation
Questions and Activities for Discussion and Application
Appendix: Writing the Research Proposal
Getting the Research Idea
Research Abstract
Formal Research Proposal
Proposal Considerations
Recommended Proposal Document Format Sequencing Layout of Proposal
References
Author Index
Subject Index
About the Author