Introduction to Test Theory

PART I: Introduction to Test Theory

Chapter 1 General Introduction Miller






Historical Context



Definition of terms



Defining Test Theory






Organization of Text



Exercises

Chapter 2 Test Development Poggio






Overview of test development process



Defining purposes of tests



Identifying behaviors



Test specifications



Item types and item construction



Item review



Item tryout



Field testing



Item analysis/reliability/validity



Preparing guidelines



Summary



Exercises

Chapter 3 Item and Test Statistics Poggio






Item Statistics for Binary Items



Item Statistics for non-Binary Items



Composites and Test Variance



Summary






Exercises

Drop chapter 4 and move the topics to other chapters as listed below:






Scales of Measurement Move to Chapter 3



Approaches to Scaling Move to Norms and Scaling Chapter

PART II: Reliability

Chapter 4 Reliability and Classical Test Theory Miller






Classical Test Theory



Reliability Index and Reliability Coefficient



Spearman-Brown and Coefficient Alpha



Standard Error of Measurement



Summary






Exercises

Chapter 5 Reliability Estimation and Applications Miller






Estimating Reliability with Two Test Administrations



Estimating Reliability with a Single Test Administration



Factors Affecting Reliability Coefficients





Test Length



Standardized Procedures



Test Variance and Sampling




Reliability of Difference Scores



Predicting True Scores



Standard Errors and Score Interpretation



Reporting Reliability



Summary






Exercises

Chapter 6 Reliability for Criterion-referenced Tests Poggio






Use of Criterion-Referenced Tests



Reliability for Domain Scores



Reliability for Mastery Classifications



Summary






Exercises

Chapter 7 Item Analysis Poggio






Item Means (Difficulty) and Item Variance



Item Discrimination



Item Analysis for Criterion-referenced Tests



Summary



Exercises

Chapter 8 Generalizability Theory Miller






Definitions and Terminology



G-studies and Estimating Variance Components



Variances and Generalizability Coefficients



D-Studies for Single Fact Design



D-Studies for Two-Facet Design



Universes with Fixed Facets



Other Issues in Generalizability



Summary






Exercises

PART III: VALIDITY

Chapter 9 Introduction to Validity






History of validity



Test Standards



Content and Validity



Correlations and Validity





Convergent and Discriminant Validity



Multitrait-Multimethod Matrices




Internal Measures of Validity



Summary



Exercises

Chapter 10 Prediction and Classification






Bivariate Regression



Multiple Regression



Partial Covariance and Correlation



Summary






Exercises

Chapter 11 Comparing Populations






Definitions and Terminology



Offensiveness: Content Review



Fair Selection Procedures



DIF: Binary Items



Summary






Exercises

Chapter 12 Factor Analysis






Models



Factor Loadings



Rotation





Orthogonal Rotations



Oblique Rotations




Communalities and Uniquenesses



Confirmatory Approaches



Summary



Exercises

PART IV: SCALES

Chapter 13 Norming and Standard Scores






Conducting a Norming Study



Probability Sampling



Types of Normative Scales



Summary






Exercises

Chapter 14 Standard Setting






Methods of Standard Setting



Uses of Standards



Summary






Exercises

Chapter 15 Item Response Theory






Definitions and Terminology



Normal Ogive Models



Logistic Models



Local Independence



Estimation of Parameters



Applications of Item Response Theory



Additional Models



Summary






Exercises

Chapter 16 Equating






Definition and Terminology



Equating Designs



Statistical Methods of Equating





Equipercentile



Linear



Equating with Item Response Theory








Summary



Exercises

Chapter 17 Future Topics