Program Evaluation

Emil J. Posavac (Ph.D., University of Illinois, Champaign) is Professor Emeritus of Psychology at Loyola University of Chicago where he served as director of the Applied Social Psychology Graduate Program and chairman of the Psychology Department. He has consulted with a number of public and private organizations. He has published over sixty papers and chapters, edited or co-edited six volumes on program evaluation and applied social psychology, and written numerous evaluation reports for health care and educational institutions. He has written a textbook (with Eugene B. Zechmeister) on statistical analysis based on emerging orientations that emphasize a more complete understanding and presentation of data. In 1990, he was awarded the Myrdal Award by the American Evaluation Association for his contributions to the advancement of program evaluation practice. Since retirement he has taught at the University of Rochester (Simon Graduate School of Business), Rochester Institute of Technology, and Trevecca Nazarene University. He teaches English as a Second Language at his church in Franklin, Tennessee.

TABLE OF CONTENTS 

 

1 Program Evaluation: An Overview  

Evaluation Tasks That Need to Be Done  

Common Types of Program Evaluations  

Activities Often Confused with Program Evaluation  

Different Types of Evaluations for Different Kinds of Programs  

Purpose of Program Evaluation  

 

2 Planning an Evaluation  

An Overview of the Core Purposes of Evaluation

Steps in Preparing to Conduct an Evaluation

Dysfunctional Attitudes Making Program Evaluation Challenging

 

3 Developing and Using a Theory of the Program

Developing a Program Theory

Evaluation Questions Flow from Program Theory

Cautions in Choosing Evaluation Criteria

Identifying  Goals and Objectives

Some Practical Limitations in Selecting Evaluation Criteria  

 

4 Developing Measures of Implementation and Outcomes

Sources of Data for Evaluation  

Gathering Information Wisely

Case Study 1: Using Multiple Measures in an Evaluation of a Summer Community Program for Youth  

Types of Measures of Evaluation Criteria  

Preparing Special Surveys  

 

5 Ethics in Program Evaluation  

Standards for the Practice of Evaluation  

Ethical Issues Involved in the Treatment of People  

Role Conflicts Facing Evaluators  

Recognizing the Needs of Different Stakeholders  

The Validity of Evaluations  

Avoiding Possible Negative Side Effects of Evaluation Procedures

Institutional Review Boards and Program Evaluation  

Ethical Problems Evaluators Report

 

6 The Assessment of Need  

Definitions of Need  

Sources of Information for the Assessment of Need  

Inadequate Assessment of Need  

Using Needs Assessments in Program Planning  

 

7 Monitoring the Implementation and Operation of Programs  

Monitoring Programs as a Means of Evaluating Programs  

What to Summarize with Information Systems  

Program Records and Information Systems  

Avoiding Common Problems in Implementing an Information System  

 

8 Qualitative Evaluation Methods  

Evaluation Settings Best Served by Qualitative Evaluations

Gathering Qualitative Information  

Case Study 2: Using Qualitative Methods in an Evaluation of a University Library  

Carrying Out Naturalistic Evaluations  

Coordinating Qualitative and Quantitative Methods  

Philosophical Assumptions  

 

9 Outcome Evaluations with One Group

One-Group Evaluation Designs  

Uses of One-Group, Descriptive Designs  

Case Study 3: A Pretest-Posttest Design to Evaluate a Peer-Based Program to Prevent Skin Cancer  

Threats to Internal Validity  

Construct Validity in Pretest-Posttest Designs  

Overinterpreting the Results of One-Group Designs  

Usefulness of One-Group Designs as Initial Approaches to Program Evaluation  

 

10Quasi-Experimental Approaches to Outcome Evaluation  

Increasing the Number of Times  Observations Are Made

Observing Other Groups

Case Study 4: Nonequivalent Control Groups Used to Evaluate an Employee Incentive Plan  

Regression-Discontinuity Design  

Observing Other Dependent Variables  

Combining Designs to Increase Internal Validity  

 

11Using Experiments to Evaluate Programs  

Experiments in Program Evaluation  

Objections to Experimentation  

The Most Desirable Times to Conduct Experiments  

Case Study 5: Teaching Doctors Communication Skills: An Evaluation with Random Assignment and Pretests  

Successfully Implementing and Interpreting an Experimental Design  

 

12Analyses of Costs and Outcomes  

Cost Analyses and Budgets  

Comparing Outcomes to Costs  

Some Details of Cost Analyses  

Case Study 6: The Value of Providing Smoking Cessation Clinics for Employees on Company Time  

Major Criticisms of Cost Analyses  

 

13Evaluation Reports: Interpreting and Communicating Findings  

Developing a Communication Plan  

Personal Presentations of Findings  

Content of Formal Written Evaluation Reports  

Provide Progress Reports and Press Releases  

 

 

References

Index