Foundations of Psychological Testing

Leslie A. Miller, PhD, SHRM-CP, PHR, has broad experience in consulting, teaching, and researching in the area of organizational and educational assessment, measurement, and development. Currently the owner of her own consulting business, LanneM TM, LLC, she provides her clients with pragmatic and affordable talent management solutions to help them acquire, develop, and retain the talent they need to achieve desired business results. Her expertise includes designing performance improvement and management tools and knowledge tests, customizing and facilitating leadership training programs, providing assessment-based executive coaching, and designing and implementing business impact evaluation and return-on-value studies. She also spends her time as an adjunct faculty member teaching organizational behavior and psychology, research, and human resources courses at Rollins College, the University of Oklahoma, and for the School of Advanced Studies at the University of Phoenix. In addition, she chairs doctoral dissertations. Previously the vice president of leadership development and human resources at the Central Florida YMCA, she was responsible for contributing to the strategic plans of the organization by leading the association’s talent management initiatives—recruiting, developing, and retaining the association’s talent. Prior to joining the YMCA, she was employed by Wilson Learning Corporation (WLC), a performance improvement company, where she served as the director of business solutions, a senior project manager, and a business solutions consultant. In these roles, she was responsible for conceptualizing, designing, managing, and implementing traditional and technology-based assessment, measurement, and training performance improvement solutions for client organizations.     Prior to joining WLC, Dr. Miller served as the assistant dean of admissions at Rollins College, where she was also a faculty member of the psychology, organizational behavior, and human resources programs. Before joining Rollins College, she was a senior research psychologist for the U.S. Department of Labor, Bureau of Labor Statistics, in Washington, D.C. At the bureau, she designed, researched, and analyzed the results of some of our nation’s most important surveys. In her current and previous roles, she has worked with various leading organizations in the high-tech, financial, pharmaceutical, and transportation industries. The holder of a doctorate in educational psychology from the University of Maryland, she has an extensive list of publications and presentations. She is a member of the APA, the Society for Industrial and Organizational Psychology, and the Society for Human Resource Management.   Robert L. Lovler, PhD, has over 30 years of experience working as both an internal and an external consultant to Fortune 500 companies in the areas of employee assessment and selection, organizational development, strategic human resource consulting, and training design and delivery. His career began at CBS Inc., where he served in several roles, including director of training for the retail consumer electronics unit, then moving up to vice president of two different units within the CBS Publishing Group. He is currently senior vice president of global human resources at Wilson Learning Corporation, an international consulting firm that focuses on human performance improvement. During his career, he has had the opportunity to design and implement a wide range of organizational interventions both domestically and internationally, working in Japan, China, South Korea, Hong Kong, England, and Italy. In the United States, he supervised the development and implementation of the assessment center used to select candidates for entry into the Environmental Protection Agency’s (EPA) Senior Executive Service Development Program, presenting the results in Washington, D.C., to Christine Todd Whitman, the EPA administrator. He also developed the system used to help select commercial airline pilots for a major U.S. airline and worked with former senator Warren Rudman to develop and implement a nationwide survey of sales practices in the rent-to-own industry. He has served as a testing consultant to the California Bar Association and the state of Pennsylvania, and he oversaw the development of the licensure examinations for medical physicists in the state of Texas. He has been on the faculty of the State University of New York at Farmingdale, and the adjunct faculty of Hofstra University, and the University of Central Florida. Dr. Lovler received his BA degree in psychology from the University of California, Los Angeles, and holds master’s and doctoral degrees from Hofstra University. He is a member of the APA and the Society for Industrial and Organizational Psychology.

Preface
About the Authors
SECTION I. OVERVIEW OF PSYCHOLOGICAL TESTING
Chapter 1. What Are Psychological Tests?
Why Should You Care About Psychological Testing?
What Is a Psychological Test?
The History of Psychological Testing
The Three Defining Characteristics of Psychological Tests
Assumptions of Psychological Tests
Test Classification Methods
Psychological Assessment, Psychological Tests, Measurements, and Surveys
Locating Information About Tests
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
Chapter 2. Why Is Psychological Testing Important?
Types of Decisions Made Using Psychological Test Results
Which Professionals Use Psychological Tests and for What Reasons?
Psychological Testing Controversies
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
Chapter 3. What Are the Ethical Responsibilities of Test Publishers, Test Users, and Test Takers?
What Are Ethics?
Professional Practice Standards
Certification and Licensure
General Responsibilities of Test Publishers, Test Users, and Test Takers
Testing Special Populations
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
SECTION II. PSYCHOMETRIC PRINCIPLES
Chapter 4. How Do Test Users Interpret Test Scores?
Levels of Measurement
Procedures for Interpreting Test Scores
Standard Scores
The Role of Norms
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
Chapter 5. What Is Test Reliability/Precision?
What Is Reliability/Precision?
Classical Test Theory
Three Categories of Reliability Coefficients
The Reliability Coefficient
Interpreting Reliability Coefficients
Factors That Influence Reliability
Generalizability Theory
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
Chapter 6. How Do We Gather Evidence of Validity Based on the Content of a Test?
Sources of Evidence of Validity
Use of Various Sources of Evidence of Validity
Evidence of Validity Based on Test Content
Face Validity
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
Chapter 7. How Do We Gather Evidence of Validity Based on Test–Criterion Relationships?
What Is Evidence of Validity Based on Test–Criterion Relationships?
Methods for Providing Evidence of Validity Based on Test–Criterion Relationships
Selecting a Criterion
Does the Criterion Measure What It Is Supposed to Measure?
Calculating and Evaluating Validity Coefficients
The Relationship Between Reliability and Validity
Using Validity Information to Make Predictions
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
Chapter 8. How Do We Gather Evidence of Validity Based on a Test’s Relation to Constructs?
The Notion of Construct Validity
Gathering Evidence of Construct Validity
Factor Analysis
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
SECTION III. DEVELOPING AND PILOTING SURVEYS AND PSYCHOLOGICAL TESTS
Chapter 9. How Do We Construct and Administer Surveys and Use Survey Data?
Similarities and Differences Between Surveys and Psychological Tests
Survey Software
The Scientific Approach to Constructing, Administering, and Using Survey Data
Survey Reliability/Precision and Validity
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
Chapter 10. How Do We Develop a Test?
Why Develop a New Test?
Defining the Testing Universe, Audience, and Purpose
Developing a Test Plan
Composing the Test Items
Writing Effective Items
Writing the Administration Instructions
Conducting the Pilot Test
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
Chapter 11. How Do We Assess the Psychometric Quality of a Test?
Conducting Quantitative Item Analysis
Conducting Qualitative Item Analysis
Revising the Test
Validating the Test
Ethical Issues Associated With Test Validation
Developing Norms and Identifying Cut Scores
Compiling the Test Manual
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
SECTION IV. USING TESTS IN DIFFERENT SETTINGS
Chapter 12. How Are Tests Used in Educational Settings?
Types of Decisions Made in the Educational Setting
Educational Professionals as Test Users
Standards Specific to Educational Professionals as Test Users
Specific Uses of Psychological Tests in Educational Settings
Norm-Referenced, Criterion-Referenced, and Authentic Assessment of Achievement
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
Chapter 13. How Are Tests Used in Clinical and Counseling Settings?
The Work of Clinical and Counseling Psychologists
Mental Health Practitioners′ Use of Psychological Tests
Case Examples of How Tests Are Used in Clinical and Counseling Settings
Psychological Testing in Depth: Assessing Autism, Depression, and Alzheimer’s Disease
Evidence-Based Assessment
The Psychological Testing Report and Client Feedback
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
Chapter 14. How Are Tests Used in Organizational Settings?
Pre-Employment Testing
Performance Appraisal
Chapter Summary
Engaging in the Learning Process
Key Concepts
Critical Thinking Questions
Appendix A: Test Spotlights
Appendix B: Guidelines for Critiquing a Psychological Test
Appendix C: Code of Fair Testing Practices in Education
Appendix D: Ethical Principles of Psychologists and Code of Conduct
Appendix E: Table of Critical Values for Pearson Product–Moment Correlation Coefficients
Glossary
References
Author Index
Subject Index