Measuring Change in Counseling and Psychotherapy

Scott T. Meier is Professor and Chair of the Department of Counseling, School, and Educational Psychology at the University at Buffalo, The State University of New York. His main research and teaching are in the areas of psychological measurement (particularly outcome assessment), research methods (program evaluation), and counseling skills (integration of case conceptualization and assessment with intervention). Dr. Meier is a member of the American Evaluation Association, the Association for Psychological Science, and the Association for Behavioral and Cognitive Therapies. He is the author or coauthor of four books (including The Elements of Counseling) and has published in American Psychologist, Journal of Counseling Psychology, Measurement and Evaluation in Counseling and Development, and the American Journal of Evaluation.

1. Introduction and Rationale

Contemporary Psychological Testing

Contemporary Psychotherapy Research and Practice

The Implications of Research Stuckness for Clinical Practice

Summary and Conclusions

2. A History of Traits

The Seeds of Conflict

The Desire to Be Scientific

The Model of Physiology

Biology and Individual Differences

The Desire to Be Relevant

The Need for Classification

The Consequences of the Adoption of a Trait-Based Measurement Paradigm

Loss of Experimental Methods Inhibits Recognition of Method Variance

The Gain of Traits and Loss of Situations

Handling Error with Classical Test Theory

Statistics Related to Measurement

Assessment as a Complement to Measurement

Deemphasizing Measurement Theory

Loss of Precision

The Wisdom and Tyranny of Tradition

The Success and Failure of the Market

Summary and Implications

3. Reliability, Validity, and Systematic Errors

Introduction

Thinking about Reliability and Validity

Types of Validity

Constructs, Theories, and Valid Measurement

Construct Explication

Multitrait–Multimethod Matrices: Investigating the Effects of Method Variance on Validity

Campbell and Fiske

Criteria for Construct Validity

An MTMM Example

Problems with Campbell and Fiske's approach

The Factor Analytic Approach to Construct Validity

History of Self-Report and Interview Errors

Self-Reports

Interviews and Observational Methods

Measurement Error

Systematic Errors Associated with Self-Reports

Dissimulation and Malingering

Social Desirability

Systematic Errors Associated with Ratings by Others

Halo Errors

Leniency and Criticalness Errors

Causes of Inconsistency

Cognitive Influences

Item Comprehension Problems

Test Cues

Low Cognitive Ability

Affective and Motivational Influences

Test Anxiety

Negative Emotional States

Environmental and Cultural Influences

Reactivity

Stereotype Threat

Summary and Implications

4. States, Traits, and Validity

Introduction

History

The Controversy of Mischel and Peterson: The Benefits of Conflict

The Rejection of Traits: Behavioral Assessment

Reinforcing the Trait Argument

Person–environment Interactions

Aptitude-by-Treatment Interactions

Environmental Assessment

Moderators of Cross-Situational Consistency

Summary and Integration

5. Context Effects and Validity

Introduction

Understanding Inconsistency: Clues from Psychophysics Measurement

The Limitations of Psychophysical Measurement

Conclusions and Implications from Psychophysical Research

Improving the Principles of Construct Explication

Test Purpose

Test Content

Test Context

Shared Contexts and Method Variance

Applications

Recommendations Related to Test Purpose

Recommendations Related to Test Content

Recommendations Related to Test Contexts

Summary and Implications

6. Nomothetic Approaches to Measuring Change and Influencing Outcomes

History and Background

Examples of Nomothetic Measures

Beck Depression Inventory

State–Trait Anxiety Inventory

Global Assessment of Functioning

Outcome Questionnaire

Psychometric Principles and Nomothetic Measures

Reliability of Nomothetic Measures

Validity

Applications

Creating Change-Sensitive Measures

Psychometric Properties of Aggregate Scales

Using Change-Sensitive Tests in Program Evaluations

An Evidence-Based Approach to Supervision

Summary and Integration

7. Idiographic Approaches to Measuring Change and Influencing Outcomes

History and Background

Psychometric Principles and Idiographic Measures

Reliability of Idiographic Measures

Validity of Idiographic Measures

Applications

Begin with the Case Conceptualizatio