Psychological Testing

Thomas P. Hogan, Ph.D., is Distinguished University Professor Emeritus in the Psychology Department, University of Scranton. He is the author/co-author of over 200 published papers, chapters, and presentations, as well as four books, on psychological testing, measurement, and methodology; and is author/co-author of several nationally used standardized tests. He has frequently served as consultant to national organizations on assessment issues. He holds a Ph.D. in psychology with specialization in psychometrics from Fordham University.

Preface iii

Part 1

1 The World of Psychological Testing, 3

Introduction, 3

Major Categories of Tests, 4

Some Additional Ways to Categorize Tests, 6

Uses and Users of Tests, 8

Major Issues: Assumptions and Questions, 10

Basic Assumptions, 10

Fundamental Questions, 12

The Differential Perspective, 13

The Historical Perspective, 13

Remote Background: Up to 1840, 14

Setting the Stage: 1840–1880, 15

The Roots: 1880–1915, 17

The Flowering: 1915–1940, 19

Consolidation: 1940–1965, 21

Just Yesterday: 1965–2000, 22

And Now: 2000–Present, 23

Major Forces, 24

The Scientific Impulse, 24

Concern for the Individual, 24

Practical Applications, 25

Statistical Methodology, 25

The Rise of Clinical Psychology, 25

Computers, 25

By Way of Definition, 28

Summary, 29

Key Terms, 30

Exercises, 30

2 Sources of Information About Tests, 32

Two Common Problems Requiring Information About Tests, 32

A Test’s Introductory Kit, 33

Comprehensive Lists of Tests, 34

Test Collection at ETS, 34

PsycTESTS, 35

Health and Psychosocial Instruments (HaPI), 35

Tests in Print, 36

Systematic Reviews, 37

Buros Reviews Available Electronically, 37

Some Other Sources of Reviews, 38

Special-Purpose Collections, 38

Books About Single Tests, 39

Textbooks on Testing, 40

Journals, 40

Publishers’ Catalogs and Personnel, 42

Other Users, 42

Strengths and Shortcomings of the Sources, 43

Summary, 44

Key Terms, 44

Exercises, 44

3 Test Norms, 47

Purpose of Norms, 47

Review of Statistics: Part 1, 48

Variables, 48

Types of Scales, 49

Organization of Raw Data, 50

Central Tendency, 51

Variability, 52

z-Scores, 53

Shapes of Distributions, 54

The Raw Score, 55

The Special Case of Theta (θ), 56

Types of Norms, 57

Percentile Ranks and Percentiles, 59

Standard Scores, 63

Developmental Norms, 69

Examples of Norm Tables, 72

Interpretive Reports and Norms, 72

Innovative Ways to Interpret Test Performance, 74

What to Read?, 74

Ready or Not?, 75

Depressed or Not?, 75

What Job for You?, 75

Conclusion, 75

Norm Groups, 76

National Norms, 76

International Norms, 76

Convenience Norm Groups, 76

User Norms, 77

Subgroup Norms, 77

Local Norms, 77

Institutional Norms, 78

Criterion-Referenced Interpretation, 79

The Standardization Group: Determining Its Usefulness, 80

Summary, 83

Key Terms, 84

Exercises, 84

4 Reliability, 86

Introduction, 86

Four Important Distinctions, 87

Review of Statistics: Part 2—Correlation and Prediction, 88

Bivariate Distribution and Correlation Coefficients, 88

Regression Line, 90

Factors Affecting Correlation Coefficients, 92

Major Sources of Unreliability, 96

Test Scoring, 96

Test Content, 98

Test Administration Conditions, 98

Personal Conditions, 99

Conceptual Framework: True Score Theory, 99

Methods of Determining Reliability, 101

Test–Retest Reliability, 101

Inter-Scorer Reliability, 102

Alternate Form Reliability, 103

Internal Consistency Reliability, 104

Split-Half Reliability, 104

Kuder–Richardson Formulas, 105

Coefficient Alpha, 106

Three Important Conclusions, 108

The Standard Error of Measurement, 108

Confidence Bands, 109

Appropriate Units for SEM, 109

Standard Errors: Three Types, 110

Some Special Issues in Reliability, 111

Reliability in Item Response Theory, 113

Generalizability Theory, 114

Factors Affecting Reliability Coefficients, 115

How High Should Reliability Be?, 115

Summary, 116

Key Terms, 117

Exercises, 117

5 Validity, 119

Introduction, 119

Refining the Definition of Validity, 120

Construct Underrepresentation and Construct-Irrelevant Variance, 121

The Basic Issue, 122

The Traditional and Newer Classifications of Types of Validity Evidence, 123

The Issue of Face Validity, 124

Content Validity, 124

Application to Achievement Tests, 124

Instructional Validity, 126

Application to Employment Tests, 127

Content Validity in Other Areas, 128

Problems with Content Validity, 128

Criterion-Related Validity, 129

External, Realistic Criterion, 130

Contrasted Groups, 132

Correlations with Other Tests, 133

Special Considerations for Interpreting Criterion-Related Validity, 134

The Reliability–Validity Relationship, 135

Combining Information from Different Tests, 138

Decision Theory: Basic Concepts and Terms, 141

Hits, False Positives, and False Negatives, 142

Base Rate, 143

Sensitivity and Specificity, 144

Construct Validity, 145

Internal Structure, 146

Factor Analysis, 146

Response Processes, 148

Effect of Experimental Variables, 148

Developmental Changes, 149

Consequential Validity, 149

Test Bias as Part of Validity, 150

The Practical Concerns, 151

Integrating the Evidence, 151

In the Final Analysis: A Relative Standard, 152

Summary, 152

Key Terms, 153

Exercises, 153

6 Test Development and Item Analysis, 155

Introduction, 155

Defining the Test’s Purpose, 156

Preliminary Design Issues, 157

Origin of New Tests, 158

Item Preparation, 159

Types of Test Items, 160

Selected-Response Items, 160

Scoring Selected-Response Items, 162

Constructed-Response Items, 162

Scoring Constructed-Response Items, 163

The Pros and Cons of Selected-Response versus Constructed-Response Items, 166

Suggestions for Writing Selected-Response Items, 167

Suggestions for Writing Constructed-Response Items, 167

Some Practical Considerations in Writing Items, 168

Technology-based Innovations in Item Structure, 169

Item Analysis, 169

Item Tryout, 170

Item Statistics, 170

Item Difficulty, 171

Item Discrimination, 171

Examples of Item Statistics, 172

Item Statistics in Item Response Theory, 174

Factor Analysis as an Item Analysis Technique, 177

Item Selection, 178

Computer-Adaptive Testing, 181

Standardization and Ancillary Research Programs, 183

Preparation of Final Materials and Publication, 184

Summary, 185

Key Terms, 185

Exercises, 185

7 Fairness and Bias, 187

Fairness: Gaining Perspective, 187

Methods of Studying Test Fairness, 190

Panel Review, 191

Differential Item Functioning, 192

Differential Prediction, 194

Measurement Invariance, 196

Accommodations and Modifications, 197

Research on Accommodations, 198

Some Tentative Conclusions about Test Fairness, 199

Summary, 200

Key Terms, 201

Exercises, 201

Part 2

8 Cognitive Abilities: Individual Tests, 205

Some Cases, 205

Introduction to Cognitive Ability Tests, 206

Some Terminology, 206

Real-World Correlates of Cognitive Abilities, 207

Structure of Cognitive Abilities, 208

Uses and Characteristics of Individual Cognitive Ability Tests, 211

Typical Items in an Individual Intelligence Test, 213

The Wechsler Scales: An Overview, 215

Historical Introduction, 215

Weschsler’s Concept of Intelligence, 216

Wechsler Adult Intelligence Scale—Fourth Edition, 216

Wechsler Intelligence Scale for Children—Fifth Edition, 223

WISC versus WAIS, 223

Psychometric Characteristics of WISC-V, 224

The Stanford-Binet, 225

Transition to New Structure: SB4 and SB5, 225

Psychometric Characteristics of SB5, 226

Brief Individually Administered Tests of Mental Ability, 227

Peabody Picture Vocabulary Test, 227

Two Other Entries, 231

A Test of a Specific Cognitive Ability: The Wechsler Memory Scale, 231

Intellectual Disability, 235

Changing Terminology, 236

The Concept of Adaptive Behavior, 236

Definition of Intellectual Disability, 237

Vineland Adaptive Behavior Scales, 238

Other Applications of Adaptive Behavior, 241

Infant and Early Childhood Tests, 241

Other Areas for Cognitive Ability Tests, 241

Trends in Individually Administered Cognitive Ability Tests, 242

Summary, 244

Key Terms, 244

Exercises, 244

9 Cognitive Abilities: Group Tests, 246

Some Cases, 246

Uses of Group-Administered Cognitive Ability Tests, 247

Common Characteristics of Group Cognitive Ability Tests, 248

Cognitive Ability Tests in School Testing Programs, 250

Otis-Lennon School Ability Test, 250

College Admissions Tests, 257

The SAT, 257

The ACT, 259

Graduate and Professional School Selection, 264

Graduate Record Examinations: General Test, 265

Military and Business Selection Tests, 269

Armed Services Vocational Aptitude Battery, 270

Wonderlic Personnel Test, 272

Culture-Fair Tests of Cognitive Ability, 273

Intelligence Tests for Microcultures, 275

Generalizations About Group Cognitive Ability Tests, 276

Summary, 277

Key Terms, 278

Exercises, 278

10 Neuropsychological Assessment, 279

Case Examples, 279

Focus on the Brain: The Road to Clinical Neuropsychology, 280

Two Main Approaches to Neuropsychological Assessment, 284

Fixed Battery Approach, 284

Luria-Nebraska Neuropsychological Battery, 284

Halstead-Reitan Neuropsychological Test Battery, 285

Flexible Battery Approach, 287

Mental Status, 288

Intelligence, 289

Achievement, 289

Attention/Concentration, 289

Language, 290

Visuospatial/Perceptual, 291

Memory, 292

Motor Functioning, 294

Executive Functions, 295

Personality/Psychological State, 296

Supplementary Information, 298

Medical History, 299

Psychiatric History, 299

Psychosocial History, 299

School Records, 300

Collateral Information, 300

Behavioral Observations, 300

Case Examples Revisited, 301

Summary, 305

Key Terms, 305

Exercises, 306

11 Achievement Tests, 307

Introduction, 307

The Ability-Achievement Continuum, 308

The Psychologist’s Interface with Achievement Tests, 308

A Broad Classification of Achievement Tests, 309

A Typical School Testing Program, 310

The Accountability Movement and Standards-Based Education, 310

Trends in Achievement Testing in the Schools, 311

Achievement Batteries, 312

Stanford Achievement Test, 312

Typical Uses and Special Features, 314

Achievement Batteries at the College Level, 315

Single-Area Achievement Tests, 316

Examples, 316

Typical Uses and Special Features, 318

Licensing and Certification Tests, 318

Examples, 319

Typical Uses and Special Features, 319

A Primer on Establishing Cutoff Scores, 320

State, National, and International Achievement Tests, 321

State Testing Programs, 321

A National Testing Program: NAEP, 322

International Testing Programs: TIMSS, PIRLS, PISA, and PIAAC, 322

Special Features, 323

Individually Administered Achievement Tests, 323

Examples, 324

Typical Uses and Special Features, 326

Curriculum Based Measures, 327

General Characteristics, 327

Examples, 327

Interpretation, 328

Some Nagging Questions about Achievement Tests, 329

Summary, 330

Key Terms, 331

Exercises, 331

12 Objective Personality Tests, 333

Introduction, 333

Uses of Objective Personality Tests, 334

A Functional Classification of Objective Personality Tests, 335

Comprehensive Inventories: Common Characteristics, 336

Specific Domain Tests: Common Characteristics, 338

The Special Problems of Response Sets and Faking, 339

Strategies for Dealing with Response Sets and Faking, 340

Major Approaches to Personality Test Development, 343

Content Method, 343

Criterion-Keying Approach, 344

Factor Analysis, 346

Theory-Driven Approach, 346

Combined Approaches, 347

Examples of Comprehensive Inventories, 347

The Edwards Personal Preference Schedule (EPPS): An Example of a Theory-Based Test, 348

The NEO Personality Inventory-3: An Example of a Factor-Analytic Test, 349

IPIP: Build Your Own Personality Inventory, 352

Specific Domain Tests, 353

Piers-Harris Children’s Self-Concept Scale, 353

Measures within Positive Psychology, 355

An Example: Subjective Well-Being, 356

Trends in the Development and Use of Objective Personality Tests, 357

Summary, 358

Key Terms, 359

Exercises, 359

13 Clinical Instruments and Methods, 360

Introduction, 360

The Clinical Interview as Assessment Technique, 361

Unstructured, Semistructured, and Structured Interviews, 361

The DSM and ICD, 362

Categorical versus Dimensional Approaches, 362

Structured Clinical Interview for DSM-IV Axis I Disorders, 363

The Employment Interview: A Sidebar, 364

Examples of Comprehensive Self-Report Inventories, 365

The Minnesota Multiphasic Personality Inventory (MMPI), 365

MMPI-2 RF (Restructured Form), 372

The Millon Clinical Multiaxial Inventory (MCMI) and the Millon Family, 373

Personality Assessment Inventory (PAI), 376

Symptom Checklist-90-R, 378

Examples of Specific Domain Tests, 380

The Beck Depression Inventory (BDI), 380

The Eating Disorder Inventory (EDI), 381

State-Trait Anxiety Inventory, 384

Behavior Rating Scales, 385

Multiscore Systems, 386

Single-Area Scales, 387

Behavioral Assessment, 387

Direct or Naturalistic Observation, 388

Analogue Behavioral Observation, 388

Behavioral Interviewing, 389

Self-Monitoring and Self-Report, 389

Cognitive-Behavioral Assessment, 390

Physiological Measures, 390

Concluding Comments on Behavioral Assessment Methods, 390

Trends in the Development and Use of Clinical Instruments, 391

Summary, 392

Key Terms, 392

Exercises, 393

14 Projective Techniques, 394

General Characteristics of Projective Techniques and the Projective Hypothesis, 394

Uses of Projective Techniques, 395

Indicators for the Use of Projectives, 397

Administration and Scoring of Projective Techniques: A Forewarning,397

The Rorschach Inkblot Test, 398

The Materials, 399

Administration and Scoring, 400

The Coding System, 402

Sequence of Scores and Structural Summary, 404

Evaluation of the Rorschach, 404

Rorschach Performance Assessment System (R-PAS), 405

Thematic Apperception Test (TAT), 406

Rotter Incomplete Sentences Blank (RISB), 408

Human Figure Drawings, 411

The Future of Projective Techniques, 413

Training of Psychologists, 413

Emergence of Managed Care, 413

Objective Scoring, Norm-Referenced Interpretation, and Psychometric

Quality, 414

Summary, 415

Key Terms, 415

Exercises, 415

15 Interests and Attitudes, 417

Introduction, 417

Orientation to Career Interest Testing, 418

Strong and Kuder, 418

Traditional Approaches, 418

Uses of Career Interest Tests, 419

A Forewarning on Names, 420

Holland Themes and the RIASEC Codes, 421

Strong Interest Inventory, 422

Kuder Career Interests Assessments, 427

Self-Directed Search (SDS), 428

Some Generalizations about Career Interest Measures, 431

Attitude Measures, 432

Likert Scales, 433

Thurstone Scales, 435

Guttman Scales, 436

Public Opinion Polls and Consumer Research, 437

Summary, 437

Key Terms, 438

Exercises, 438

16 Ethical and Legal Issues, 439

Ethics versus Law, 439

Ethical Issues, 440

Background on Professional Ethics, 440

Sources of Ethical Principles for Testing, 441

Generalizations about Ethical Use of Tests, 444

Competence, 444

Informed Consent, 444

Knowledge of Results, 445

Confidentiality, 445

Test Security, 445

Test Development and Publication, 445

Automated Scoring/Interpretation Systems, 446

Unqualified Persons, 446

Test User Qualifications, 446

Legal Issues, 447

Areas of Application: An Overview, 448

Definition of Laws, 448

Laws Related to Testing, 449

The Fourteenth Amendment, 450

The Civil Rights Acts of 1964 and 1991, 451

Rehabilitation Act of 1973 and Americans with

Disabilities Act of 1990, 451

The Handicapped/Disabled in Education: P.L. 94–142 and IDEA, 452

FERPA and HIPAA, 453

EEOC Guidelines, 454

ESEA, NCLB, and ESSA, 456

Illustrative Court Cases, 456

Griggs v. Duke Power, 457

Debra P. v. Turlington and GI Forum v. TEA, 458

Larry P. v. Riles, PASE v. Hannon, and Crawford v. Honig, 459

Karraker v. Rent-A-Center, 460

Atkins v. Virginia and Hall v. Florida, 460

New Haven Firefighters Case, 461

Forensic Application of Tests, 461

Two Legal Terms, 462

Three Areas of Special Concern, 462

And Beyond, 463

Some Generalizations about the Interface of Testing and the Law, 463

Summary, 464

Key Terms, 465

Exercises, 465

Appendix A. Test Reviewing and Selection 467

Appendix B. How to Build a (Simple) Test 473

Appendix C. Contact Information for Major Test Publishers 479

Appendix D. Sample Data Sets 480

Appendix E. Answers to Selected Exercises 481

Glossary 485

References 501

Name Index 523

Subject Index 529