Operational Characteristics of Adaptive Testing Procedures Using the Graded Response Model
Abstract
The purpose of the present research was to develop general guidelines to assist practitioners in setting up operational computerized adaptive testing (CAT) sys tems based on the graded response model. Simulated data were used to investigate the effects of systematic manipulation of various aspects of the CAT procedures for the model. The effects of three major variables were examined: item pool size, the stepsize used along the trait continuum until maximum likelihood estima tion could be calculated, and the stopping rule em ployed. The findings suggest three guidelines for graded response CAT procedures: (1) item pools with as few as 30 items may be adequate for CAT; (2) the variable-stepsize method is more useful than the fixed- stepsize methods; and (3) the minimum-standard-error stopping rule will yield fewer cases of nonconverg ence, administer fewer items, and produce higher cor relations of CAT θ estimates with full-scale estimates and the known θs than the minimum-information stop ping rule. The implications of these findings for psy chological assessment are discussed. Index terms: computerized adaptive testing, graded response model, item response theory, polychotomous scoring.
Get full access to this article
View all access and purchase options for this article.
References
Andrich, D. (1978a). Application of a psychometric rating model to ordered categories which are scored with successive integers. Applied Psychological Measurement, 2, 581-594.
Andrich, D. (1978b). A rating formulation for ordered response categories . Psychometrika, 43, 561-573.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick, Statistical theories of mental test scores. Reading MA: Addison-Wesley.
Bock, R.D. (1972). Estimating item parameters and latent abilities when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.
Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459.
De Ayala, R.J., & Koch, W.R. (1987, April). Computerized adaptive testing: A comparison of the nominal response model and the three-parameter logistic model. Paper presented at the annual meeting of the National Council on Measurement in Education, Washington DC.
Dodd, B.G. (1985). Attitude scaling: A comparison of the graded response and partial credit latent trait models (Doctoral dissertation, University of Texas at Austin, 1984). Dissertation Abstracts International, 45, 2074A.
Dodd, B.G. (1987, April). Computerized adaptive testing with the rating scale model. Paper presented at the Fourth International Objective Measurement Workshop, Chicago.
Koch, W.R. (1983). Likert scaling using the graded response latent trait model. Applied Psychological Measurement, 7, 15-32.
Koch, W.R., & Dodd, B.G. (1985, April). Computerized adaptive attitude measurement . Paper presented at the annual meeting of the American Educational Research Association, Chicago.
Koch, W.R., & Dodd, B.G. (in press). An investigation of procedures for computerized adaptive testing using partial credit scoring. Applied Measurement in Education.
Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
McKinley, R.L., & Reckase, M.D. (1983). An evaluation of one- and three-parameter logistic tailored testing procedures for use with small item pools (Research Report ONR83-1). Iowa City IA: American College Testing Program.
Reckase, M.D. (1981). Final report: Procedures for criterion referenced tailored testing. Columbia MO: University of Missouri.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, No. 17.
Samejima, F. (1976). Graded response model of the latent trait theory and tailored testing. In C. K. Clark (Ed.), Proceedings of the First Conference on Computerized Adaptive Testing. Washington DC: U.S. Government Printing Office.
Thissen, D. (1986). MULTILOG Version 5 user's guide. Mooresville IN: Scientific Software, Inc.
Urry, V.W. (1977). Tailored testing: A successful application of latent trait theory. Journal of Educational Measurement, 14, 181-196.
Weiss, D.J. (1981). Final report: Computerized adaptive ability testing . Minneapolis: University of Minnesota .
Weiss, D.J. (1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6, 473-492.
Weiss, D.J. (1983). Final report: Computer-based measurement of intellectual capabilities. Minneapolis: University of Minnesota.
Weiss, D.J. (1985). Final report: Computerized adaptive measurement of achievement and ability. Minneapolis: University of Minnesota.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published: June 1989
Issue published: June 1989
Authors
Metrics and citations
Metrics
Journals metrics
This article was published in Applied Psychological Measurement.
VIEW ALL JOURNAL METRICSArticle usage*
Total views and downloads: 87
*Article usage tracking started in December 2016
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 49 view articles Opens in new tab
Crossref: 49
- Simulating computerized adaptive testing in special education based on...
- Development and Use of Anchoring Vignettes: Psychometric Investigation...
- Development of testlet instruments to measure science process skills o...
- An Investigation of Exposure Control Methods With Variable-Length CAT ...
- Variable-Length Stopping Rules for Multidimensional Computerized Adapt...
- Comparing computer adaptive testing stopping rules under the generaliz...
- Projection-Based Stopping Rules for Computerized Adaptive Testing in L...
- Development and Validation of the Activities of Daily Living Short For...
- Item-level psychometrics of the ADL instrument of the Korean National ...
- Computer‐Based Testing and the Internet
- Investigating Constraint-Weighted Item Selection Procedures in Unfoldi...
- Assessment of the psychometrics of a PROMIS item bank: self-efficacy f...
- The Performance of the Modified Multidimensional Priority Index for It...
- A Polytomous Item Response Theory Analysis of Social Physique Anxiety ...
- Comparing the Performance of Five Multidimensional CAT Selection Proce...
- Single-item screens identified patients with elevated levels of depres...
- A New Stopping Rule for Computerized Adaptive Testing
- A Comparison of Anchor-Item Designs for the Concurrent Calibration of ...
- Efficiency of static and computer adaptive short forms compared to ful...
- Differential item functioning was negligible in an adaptive test of fu...
- Screening for Elevated Levels of Fear-Avoidance Beliefs Regarding Work...
- SIMPOLYCAT: An SAS program for conducting CAT simulation based on poly...
- Item difficulty parameter estimation using the idea of the graded resp...
- A Computerized Adaptive Test for Patients With Hip Impairments Produce...
- Computerized adaptive test for patients with knee impairments produced...
- Item Selection Strategies for Computerized Adaptive Testing with the G...
- Computerized Adaptive Testing of Personality Traits
- Computerized Adaptive Testing for Polytomous Motivation Items: Adminis...
- Simulated computerized adaptive test for patients with lumbar spine im...
- Simulated computerized adaptive test for patients with shoulder impair...
- Applying Bayesian Item Selection Approaches to Adaptive Tests Using Po...
- A Randomized Experiment to Compare Conventional, Computerized, and Com...
- Graded Response Model
- An examination of the psychometric properties of the physical self-des...
- Postscript: Applications of Rasch Analysis in Health Care
- Precision of Warm’s Weighted Likelihood Estimates for a Polytomous Mod...
- Equating Health Status Measures With Item Response Theory
- A Comparison of Maximum Likelihood Estimation and Expected a Posterior...
- The Effect of Population Distribution and Method of Theta Estimation o...
- An Investigation of Procedures for Computerized Adaptive Testing Using...
- Computerized Adaptive Testing With Polytomous Items
- The Influence of Multidimensionality on the Graded Response Model
- Computerized Adaptive Testing Using the Partial Credit Model: Effects ...
- The Nominal Response Model in Computerized Adaptive Testing
- Adapting Marketing Surveys to Individual Respondents
- A Comparison of the Partial Credit and Graded Response Models in Compu...
- The Effect of Item Selection Procedure and Stepsize on Computerized Ad...
- Adaptive Designs for Likert-Type Data: An Approach for Implementing Ma...
- Parameter Recovery in the Graded Response Model Using MULTILOG
Figures and tables
Figures & Media
Tables
View Options
Get access
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
