Psychological Testing

Theresa J.B. Kline (Ph.D., 1990) is a Professor of Industrial-Organizational Psychology at the University of Calgary.  She has an active research program in the area of psychometrics, team performance, organizational effectiveness, and work attitudes.  Theresa has published two books on teams, Teams that Lead (2003) and Remaking Teams (1999), and over 50 peer-reviewed journal articles. Theresa teaches psychometrics, statistics, methods and organizational psychology at the undergraduate and graduate level.  She has an active organizational consulting practice, with projects ranging from individual and organizational assessment to strategic alignment. She has made presentations and run workshops on topics that range from statistics and methods, to how to use assessment tools for personnel decision-making in an ethical manner, to executive team development practices. Theresa has supervised nine master’s theses and eight doctoral dissertations.  She encourages her students to take an active role in deciding what they will study and, as a result, she has learned at least as much from them they have from her; her pupils have drawn her into research areas such as organizational learning, lifelong learning, organizational citizenship, fairness, leadership, job change, workplace stress, and performance appraisal.

Preface
The Assessment of Individuals: The Critical Role and Fundamentals of Measurement
Measurement in the Physical Sciences
Measurement in the Social Sciences
Historical Highlights of Measurement
Statistics Background
The First Step: Identifying the Construct
Links Between Constructs
Construct Cleanliness
Single versus Multiple Constructs
Summary and Next Step
Problems and Exercises
Designing and Writing Items
Empirical, Theoretical and Rational Approaches to Item
Empirical, Theoretical and Rational Approaches to Item
Literature Search
Subject Matter Experts
How Many Items?
Attitudinal Items: Early Work in Item Generation
Assessing Behaviors
Pilot Testing
Summary and Next Step
Problems and Exercises
Designing and Scoring Responses
Open-Ended Responses
Closed-Ended Questions
Example 1: Proportional Differences for a Single Variable
Example 2: Proportional Differences for Two Variables
Dichotomous Responses
Multiple Choice Tests
Continuous Responses
Ipsative versus Normative Scales
Difference and Change Scores
Summary and Next Step
Problems and Exercises
Collecting Data: Sampling and Screening
Probability Sampling
Non-Probability Sampling
Sample Sizes
Missing Data
Preparing to Analyze Your Data
Summary and Next Step
Problems and Exercises
Classical Test Theory: Assumptions, Equations, Limitations and Item Analyses
Classical Test Theory
Theory of True and Error Scores: Description and Assumptions
Ramifications and Limitations of Classical Test Theory Assumptions
Item Analysis within CTT: Approaches, Statistical Analyses, and Interpretation
Descriptive Statistics
Summary
Problems and Exercises
Modern Test Theory: Assumptions, Equations, Limitations and Item Analyses
Modern Test Theory
Models
One-Parameter Logistic Model
Two-Parameter Logistic Model
Three-Parameter Logistic Model
Multiple-response IRT Models
Parameter Estimation
Scoring Respondents
Model Fit
Assumptions
Ramifications of the Assumptions of Modern Test Theory
Practical Advantages of Modern Test Theory
Limitations of Modern Test Theory
Computer Programs
Practical Considerations
Summary
Next Steps
Problems and Exercises
Reliability of Test Scores and Test Items
Test-retest Reliability
Alternative Forms Reliability
Measures of Internal Consistency
Setting Confidence Intervals
Reliability of a Composite
Difference Scores - A Reliability Concern
Practical Questions
Summary and Next Steps
Problems and Exercises
Reliability of Raters
Inter-Rater Reliability Indices
Reliability Generalization
Modern Test Theory Approaches to Reliability
Summary and Next Steps
Problems and Exercises
Assessing Validity Using Content and Criterion
Methods
Asking the Test Takers
Asking the Subject Matter Experts
Assessments Using Correlation and Regression: Criterion-Related Studies
Classification Approaches to Test Score Validation
Group Differences and Test Bias
Extending the Inferences of Criterion-Related Validity Studies
Summary
Problems and Exercises
Assessing Validity Via Item Internal Structure
Principal Components Analysis
Common Factor Analysis
Common Factor Analysis using Analysis of Covariance
Structures
Some Other Issues in Factor Analysis
Practical Issues
Concluding Comments on Internal Structure and Validity
Threats to the Validity of Scores
Multitrait-Multimethod Assessment
Closing Comments on Test Score Validity
Summary
Problems and Exercises
Ethics and Professional Issues in Testing
Professional Standards and Guidelines
Ethical Procedures and Protocols
Test Administration
Integrity Testing
Computerized Testing
Coaching, Testwiseness, and Re-takes
Testing Legislation
Test Item Bias and Adverse Impact
Translation Issues
Electronic Presentation and Capture
Summary
Problems and Exercises
Brief Reviews of Some Selected Tests and Concluding Comments
Information about Existing Tests
Some Intelligence Tests
Academic Achievement Tests
Structured Personality Tests
Career Interest/Guidance Instruments
Chapter Summary
A Quick Book Review
Concluding Comments
Problems and Exercises
Appendices
References
Index