Research Methods in Practice

Dahlia K. Remler is Professor at the School of Public Affairs, Baruch College, and the Department of Economics, Graduate Center, both of the City University of New York. She is also a Research Associate at the National Bureau of Economic Research. Dahlia has been in an unusual mix of disciplinary and interdisciplinary settings. She received a BS in electrical engineering from the University of California at Berkeley, a DPhil in physical chemistry from Oxford University—while a Marshall Scholar—and a PhD in economics from Harvard University. During the Clinton administration’s health care reform efforts, Dahlia held a fellowship at the Brookings Institution to finish her dissertation on health care cost containment. She then held a postdoctoral research fellowship at Harvard Medical School, followed by assistant professorships at Tulane’s and Columbia’s Schools of Public Health, prior to joining the faculty at Baruch. She enjoys comparing and contrasting how different disciplines see the same issues. Dahlia has published widely in a variety of areas in health care policy, including health care cost containment, information technology in health care, cigarette tax regressivity, simulation methods for health insurance take-up, and health insurance and health care markets. She has also recently started working on higher education and media issues. Her work has appeared in the Journal of Policy Analysis and Management, Health Affairs, the Quarterly Journal of Economics, the American Journal of Public Health, Medical Care Research and Review, and many other journals. She blogs on health care policy, higher education and other topics at DahliaRemler.com. Dahlia lives with her husband, Howard, in New York City, where they enjoy the city’s theaters, restaurants, and parks—and Dahlia enjoys being a complete amateur dancer in some of the city’s superb dance studios. Gregg G. Van Ryzin is Professor at the School of Public Affairs and Administration, Rutgers University—Newark. He received his BA in geography from Columbia University and his PhD in psychology from the City University of New York. During his doctoral training, he worked as a planner for a nonprofit housing and community development organization in New York City, and he completed his dissertation on low income housing for the elderly in Detroit. He next worked in Washington, D.C., for ICF Inc. and later Westat, Inc. on surveys and program evaluations for the U.S. Department of Housing and Urban Development and other federal agencies. In 1995, he joined the faculty of the School of Public Affairs, Baruch College, where he directed their Survey Research Unit for 8 years. In that role, he helped develop and direct the New York City Community Health Survey, a large-scale behavioral health survey for the city’s health department, and also played a key role in shaping and conducting the city’s survey of satisfaction with government services. He has spent time in Madrid, collaborating with researchers there on the analysis of surveys about public attitudes toward Spanish government policy. Gregg has published many scholarly articles on housing and welfare programs, survey and evaluation methods, and public opinion about government services and institutions. His work has appeared in the International Review of Administrative Sciences, the Journal of Policy Analysis and Management, the Journal of Public Administration Research and Theory, the Journal of Urban Affairs, Nonprofit and Voluntary Sector Quarterly, Public Administration Review, Public Management Review, Public Performance and Management Review, Urban Affairs Review, and other journals. Gregg lives in New York City with his wife, Ada (a history professor at NYU), and their daughters Alina and Lucia. They enjoy life in their Greenwich Village neighborhood, escaping on occasion to Spain, Miami, Maine, Cuba, and other interesting places in the world.

Preface
Acknowledgments
About the Authors
PART I: FOUNDATIONS
Chapter 1. Research in the Real World
Learning Objectives
Do Methods Matter?
Research, Policy, and Practice
Evidence Can Mislead
What Is Research?
Descriptive and Causal Research
Epistemology: Ways of Knowing
Approaching Research From Different Angles
Ethics of Research
Conclusion: The Road Ahead
Exercises
Chapter 2. Theory, Models, and Research Questions
Learning Objectives
Community Policing Comes to Portland
What Is a Theory?
What Is a Model?
Logic Models: Mechanisms of Programs
Alternative Perspectives on Theory in Social Research
How to Find and Focus Research Questions
Conclusion: Theories Are Practical
Chapter 3. Qualitative Research
Learning Objectives
Fighting Malaria in Kenya
What Is Qualitative Research?
Existing Qualitative Data
Qualitative Interviews
Focus Groups
Qualitative Observation
Participant Observation and Ethnography
Case Study Research
Qualitative Data Analysis
The Qualitative-Quantitative Debate
Ethics in Qualitative Research
Conclusion: Matching Methods to Questions
Exercises
PART II: STRATEGIES FOR DESCRIPTION
Chapter 4. Measurement
Learning Objectives
The U.S. Poverty Measure
What Is Measurement?
Conceptualization
Operationalization
Validity
Criterion-Related Validity
Measurement Error
Reliability
Validity and Reliability in Qualitative Research
Levels of Measurement
Measurement in the Real World: Trade-offs and Choices
Conclusion: Measurement Matters
Exercises
Chapter 5. Sampling
Learning Objectives
Gauging the Fallout From Hurricane Katrina
Generalizability
Basic Sampling Concepts
Problems and Biases in Sampling
Nonprobability Sampling
Random (Probability) Sampling
Sampling Distributions, Standard Errors, and Confidence Intervals
Sampling in Practice
Sampling and Generalizability: A Summary
Exercises
Chapter 6. Secondary Data
Learning Objectives
Tracking a Global Pandemic
Quantitative Data Forms and Structures
Administrative Records
Aggregate Data Tables
Public Use Microdata
Secondary Qualitative Data
Big Data
Linking Data
Some Limitations of Secondary Data
Conclusion
Exercises
Chapter 7. Surveys and Other Primary Data
Learning Objectives
Taking the Nation’s Economic Pulse
When Should You Do a Survey?
Steps in the Survey Research Process
Modes of Survey Data Collection
Crafting a Questionnaire
Ethics of Survey Research
Other Ways to Collect Primary Data
Conclusion
Exercises
PART III: STATISTICAL TOOLS AND INTERPRETATIONS
Chapter 8. Making Sense of the Numbers
Learning Objectives
“Last Weekend I Walked Eight”
Units, Rates, and Ratios
Statistics Starting Point: Variables in a Data Set
Distributions
Measures of Center: Mean and Median
Measures of Spread and Variation
Relationships Between Categorical Variables
Relationships Between Quantitative Variables: Scatterplots and Correlation
Simple Regression: Best-Fit Straight Line
Practical Significance
Statistical Software
Conclusion: Tools for Description and Causation
Exercises
Chapter 9. Making Sense of Inferential Statistics
Learning Objectives
But Is It Significant?
Statistical Inference: What’s It Good For?
The Sampling Distribution: Foundation of Statistical Inference
Confidence Intervals
Significance Tests
Statistical Significance, Practical Significance, and Power
Issues and Extensions of Statistical Inference
Conclusion
Exercises
Chapter 10. Making Sense of Multivariate Statistics
Learning Objectives
Multiple Regression: The Basics
Inference for Regression
Categorical Independent Variables
Interactions in Regression
Functional Form and Transformations in Regression
Categorical Variables as Dependent Variables in Regression
Which Statistical Methods Can I Use?
Other Multivariate Methods
Conclusion
Exercises
PART IV: STRATEGIES FOR CAUSATION
Chapter 11. Causation
Learning Objectives
Family Dinners and Teenage Substance Abuse
Alternative Explanations of a Correlation
Causal Mechanisms
Evidence of Causation: Some Critical Clues
Self-Selection and Endogeneity
The Counterfactual Definition of Causation
Experimentation and Exogeneity: Making Things Happen
Conclusion: Tools to Probe Causation
Exercises
Chapter 12. Observational Studies
Learning Objectives
Private Versus Public Schools
What Is an Observational Study?
Control Variables
Matching
Control Variables: An Empirical Example
How to Choose Control Variables
Epidemiological Approaches to Observational Studies
Conclusion: Observational Studies in Perspective
Exercises
Chapter 13. Using Regression to Estimate Causal Effects
Learning Objectives
Cigarette Taxes and Smoking
From Stratification to Multiple Regression
Does Greenery Affect Birth Outcomes?
Further Topics in Regression for Estimating Causal Effects
Control Variables With Exogenous Independent Variables: The Gender Earnings Gap
Other Multivariate Techniques for Observational Studies
Conclusion: A Widely Used Strategy, With Drawbacks
Exercises
Chapter 14. Randomized Experiments
Learning Objectives
Time Limits on Welfare
Random Assignment: Creating Statistical Equivalence
The Logic of Randomized Experiments: Exogeneity Revisited
The Settings of Randomized Experiments
Generalizability of Randomized Experiments
Variations on the Design of Experiments
Artifacts in Experiments
Analysis of Randomized Experiments
Ethics of Randomized Experiments
Qualitative Methods and Randomized Experiments
Conclusion: A Gold Standard, With Limitations
Exercises
Chapter 15. Natural and Quasi Experiments
Learning Objectives
A Casino Benefits the Mental Health of Cherokee Children
What Are Natural and Quasi Experiments?
Internal Validity of Natural and Quasi Experiments
Generalizability of Natural and Quasi Experiments
Types of Natural and Quasi Experimental Studies
Difference-in-Differences Strategy
Instrumental Variables and Regression Discontinuity
Regression Discontinuity
Ethics of Quasi and Natural Experiments
Conclusion
Exercises
PART V: CONTEXT AND COMMUNICATION
Chapter 16. The Politics, Production, and Ethics of Research
Learning Objectives
Risking Your Baby’s Health
From Research to Policy
The Production of Research
Making Research Ethical
Making Research Open and Transparent
Conclusion
Exercises
Chapter 17. How to Find, Review, and Present Research
Learning Objectives
Where to Find Research
How to Search for Studies
How to Write a Literature Review
How to Communicate Your Own Research
How to Publish Your Research
Conclusion
Exercises
Glossary
References
Index