Computerized Adaptive Testing
From Concept to Implementation
David J. Weiss and Alper Şahin
- What is CAT?
- Benefits of CAT
- Challenges of CAT
- When is CAT Feasible and When is it Not?
2. The Birth of Adaptive Testing and Its Current Status
- Binet’s Adaptive Test
- The Evolution of Adaptive Testing into CAT
- Early Approaches to CAT
- The Current Status of Applications of CAT
3. Item Response Theory Used in CAT
- Item Response Functions for Dichotomously Scored Items
- Option Response Functions for Polytomously Scored Items
- Information Functions
- Measuring People with IRT
- Estimating IRT Item Parameters
- Summary
4. Using CAT to Measure Individual Differences
- The Components of a Measurement CAT
- Putting it All Together: How CAT Works
- Some Properties of Fully Adaptive IRT-Based CATs
- Advanced Issues
5. CAT for Classification and Classification Plus Measurement
- Setting the Cutscore
- Types of Classification Applications
- Advanced Issues
II. Getting Your CAT Operational
6. Feasibility and Planning
- Developing a Business Case
- The Item Bank
- Costs of Test Delivery
- Potential Risks
- Non-Psychometric Risks
- Additional Considerations
- Conclusions
7. Developing a CAT Item Bank: Writing and Evaluating Items
- How Many Items Do I Need in My Bank?
- What Kinds of Items Work for CAT?
- Developing High-Quality Items and Item Banks
- Characteristics of a Good Item Bank
- Further Information
8. Pretesting and Linking Your Items, and Adding New Items to an Existing Bank
- Pretesting Your Items
- Creating Your Data Matrix
- IRT Item (and Person) Parameter Linking
- Additional Considerations in Item Bank Development
- Advanced Topic: Automated Item Generation
9. Using Simulations to Design a CAT
- Post-Hoc or Real-Data Simulation
- Designing a CAT from a CAT Item Bank
- Hybrid Simulations
- Conclusions
10. Implementing Your CAT
- CAT Delivery Software
- How Can I Get CAT Operational for My Organization?
III. Applications, Special Types of CATs, Special Issues, and a Potential Future for CAT
11. Examples of Operational CAT Programs
- CAT in Education
- CAT for Military Accessioning
- CAT for Pre-Employment Testing
- CAT for Certification and Licensure
- CAT in Healthcare
- Additional Applications of CAT
12. Other Types of CATs
- Multistage Tests
- CAT for Cognitive Diagnosis
- CAT for Measuring Multiple Variables
13. Special Issues
- Constrained CATs
- CAT Using Items with Multiple Questions Per Stimulus
- CAT Administration Issues
- The Examinee Experience
14. Adaptive Measurement of Change: Evaluating the Psychometric Significance of Measured Individual Change
- Identifying Significant Individual Change Using an Early Adaptive Measurement of Change (AMC) Method
- The Likelihood Ratio Index (LRI)
- Example AMC Applications
- Multivariate (Profile) Change
- AMC Research
- Discussion and Conclusions
15. Some Pathways Toward Future CATs and CAT Research
- Item Banking
- Test Delivery
- Supplementing and Improving Scores
- Integration of Testing and Instruction
- A Path for Future CAT Research: Does CAT Improve Validity?
- Leveraging Current and Future Technology to Improve Testing
Software Appendices
1. Software Packages for Estimating IRT Parameters
2. Software Packages for CAT Simulations
3. Software Platforms for CAT Delivery
References
Index