Evaluating Practice

Preface  

Prologue  

 

Part I 

What Are You Getting Into? 


1  Integrating Evaluation and Practice


 

Introduction to Single-System Designs  

What Are Single-System Designs? 


Evidence-Based Practice

Single-System Designs and Classical Research: The Knowledge-Building Context 


Single-System Evaluation, Qualitative Research, and Quantitative Research 


Advantages of Using Single-System Designs in Practice 


A Walk through the Evaluation Process 


Summary 


 

 

Part IIConceptualizing and Measuring Targets and Objectives/Goals  


2  Basic Principles of Conceptualization and Measurement


 

Introduction 


What Is Conceptualization? 


What Is Measurement? 


Definition as a First Step in Measurement 


Can Everything Be Measured? 


Key Characteristics of All Measures 


Summary 


 

3   Specifying Problems and Goals  


 

Introduction: From General Problems to Specific Targets of Intervention 


Specifying Client Concerns: Identifying and Clarifying Problems and Potentials 


Specifying Goals and Objectives


Using Goal Attainment Scaling (GAS) to Establish Goals 


Setting Goals in Groups 


Problems and Issues in Setting Goals 


Summary 


 

4  Developing a Measurement and Recording Plan


 

Introduction 


Steps in Developing a Recording Plan 


Charting: Putting Your Information on Graphs 


Problem-Oriented Records (POR)


Use of Computers in Evaluation and Practice 


Summary 


 

Computer Assisted Social Services (CASS): A User’s Guide 


Appendix: Installing CASS 



5   Behavioral Observation  


 

Introduction 


General Guidelines for Behavioral Observation 


Sampling Behaviors 


Instruments for Recording Behaviors 


Ensuring Accurate Observations 


Methods of Recording Behavior 


Analog Situations 


Recording Behavior in Groups 


Summary 


 

6    Individualized Rating Scales  


 

Introduction 


Uses of Individualized Rating Scales 


Constructing and Using Individualized Rating Scales 


Summary 


 

7     Standardized Scales


 

Introduction 


What Are Standardized Scales? 


Selecting a Standardized Scale 


Administering a Standardized Scale 


Some Available Standardized Self-Report Scales 


Some Available Standardized Scales for Practitioners 


Some Available Standardized Scales for Relevant Others 


Some Available Standardized Scales for Independent Observers 


Do-It-Yourself Scales 


Using Standardized Scales in Groups 


Computer Management of Standardized Scales 


Summary 


Appendix 




Computer Assisted Assessment Package (CAAP): A User’s Guide 



 

8  Logs


 

Introduction 


Types of Client Logs 


Putting Qualitative and Quantitative Information Together 


Introducing Clients to Logs 


Practitioner Logs


Maximizing and Verifying the Reliability and Validity of Logs 


Summary 


 

9     Reactivity and Nonreactive Measures


 

Introduction 


Reactivity of Measures 


Unobtrusive (Nonreactive) Measures 


Summary 


 

10    Selecting a Measure  


 

Introduction

Considerations in Deciding on a Measure 


Use of Multiple Measures 


Selecting a Measure 


Summary 


 

Part III

Evaluation Designs  


11    Basic Principles of Single-System Designs  

 

Introduction 


An Example Connecting Practice and Evaluation Designs 


Purposes of Single-System Designs


Unique Characteristics of Single-System Designs 


Causality in Single-System Designs 


External Validity and Generalizability 


Overview of Single-System Designs

Summary 


 

12    Baselining


 

Introduction 


Purposes of the Baseline 


Types of Baselines 


How Long Should Baselining Continue? 


When Are Baselines Not Necessary? 


Issues Regarding Baselining 


Summary


 

13    From the Case Study to the Basic Single-System Design: A-B  

 

Introduction 


Case Studies or Predesigns


Design A-B: The Basic Single-System Design 


Summary 


 

14    The Experimental Single-System Designs: A-B-A, A-B-A-B, B-A-B


 

Introduction


Basic Experimental Designs 


Summary 


 

15    Multiple Designs for Single Systems


 

Introduction 


Multiple-Baseline Designs: Across Problems, Clients, or Settings 


Multiple-Target Designs


Variations on Multiple Designs 


Summary 


 

16   Changing Intensity Designs and Successive Intervention Designs


 

Introduction 


Changing Intensity Designs: A-B1-B2-B3 


Successive Intervention Designs: A-B-C, A-B-A-C, A-B-A-C-A 


Summary 


 

17    Designs for Comparing Interventions


 

Introduction 


Alternating Intervention Design: A-B/C-(B orC) 


Interaction Design: A-B-A-B-BC-B-BC 


Summary 


 

18   Selecting a Design


 

Introduction


Framework for Selecting a Design 


Needed: A Design for All Seasons 


Creativity in Single-System Designs: Making Your Own Designs 


Evaluation in Minimal-Contact Situations 


Single-System Designs in Managed Care: The Stretch Design 


Trouble-Shooting: “Okay, I Understand Everything That You Said, but My Case Is Different.” 


Summary 


 

Part IV

Analyzing Your Results  


19   Basic Principles of Analysis


 

Introduction 


Distinguishing Effort, Effectiveness, and Efficiency 


Significance—Practical, Statistical, and Theoretical 


Evaluating Goal Achievement 


Issues in Analysis of Data 


Computer Analysis of Data for Single-System Designs 


The Issue of Autocorrelation 


Tools in Analysis of Data 


Summary 


 

20   Visual Analysis of Single-System Design Data


 

Introduction 


Definition of Terms 


Basic Patterns and Implications 


Visual Inspection of Raw Data 


Interpreting Ambiguous Patterns 


Problems of Visual Inspection 


Creating a Chart with SINGWIN 


Summary 


 

21    Descriptive Statistics


 

Introduction 


Measures of Central Tendency 


Measures of Variation 


Using Measures of Central Tendency and Variability to Graph Group Data


Computing and Graphing Measures of Central Tendency and Variation with SINGWIN 


Measures of Trend


Measures of Effect Size 


Optimal Uses and Cautions for Specific Descriptive Statistics 


Summary 


 

22    Tests of Statistical Significance for Single-System Designs


 

Introduction 


Proportion/Frequency Approach 


Three-Standard-Deviation-Band Approach (X-Moving-Range-Chart) 


Chi-Square 


Conservative Dual-Criteria (CDC) Approach 


t-Test 


General Considerations in Using Tests of Statistical Significance 


Optimal Uses and Cautions for Specific Analytic Procedures 


Summary 


 

23     Computer Analysis of Single-System Design Data


 

Chapter Overview 


Starting SINGWIN 


Exiting SINGWIN 


Getting the Big Picture 


Using Specific Procedures 


Appendix: Installing Singwin 


Summary 


 

24    Selecting a Procedure for Analyzing Data


 

Introduction 


Framework for Selecting a Procedure for Analyzing Data 


Other Statistical Considerations 


Nonstatistical Considerations 


Limitations 


Summary 


 

Part V

The Challenge of Single-System Designs


25    Not for Practitioners Alone  

 

Introduction 


Special Applications of Single-System Designs 


Recent Criticisms of Single-System Evaluation 


For the Client 


For the Administrator 


For Educators and Students 


Summary 


 

References


Name Index


Subject Index