![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
This paper aims to translate for practitioners the principles and methods for evaluating screening measures in education, including benchmark goals and cut points, from our technical manuscript “Evaluation of Diagnostic Systems: The Selection of Students at Risk of Academic Difficulties” (this issue). We offer a brief description of procedures developed over the past 50 years including the receiver operating characteristic (ROC) curves, the area under the ROC curve as a general measure of screener accuracy, and approaches to selecting a specific cut score to indicate risk. We also provide reporting standards to help practitioners evaluate research on screeners supported by best practices and to encourage researchers to attend to key reporting principles, such as using confidence bounds as estimates of precision. We then discuss examples from the literature and emphasize the imprecision of statistical estimates from small samples. Screeners and diagnostic tests, developed and evaluated with care and implemented consistently in schools, can improve educators’ decisions about resource allocation and ultimately improve the delivery of supports to students.
References
|
Atkinson, R. C. (1963). A variable sensitivity theory of signal detection. Psychological Review, 70, 91–106. Google Scholar | Crossref | Medline | ISI | |
|
Coker, D. L., Ritchey, K. D. (2014). Universal screening for writing risk in kindergarten. Assessment for Effective Intervention, 39, 245–256. Google Scholar | SAGE Journals | |
|
Missall, K., Hosp, J. (2013). Goal setting for grade-level standards in Iowa: K–6 reading. Iowa City: The University of Iowa. Google Scholar | |
|
Smolkowski, K., Cummings, K. (2015). Evaluation of the DIBELS (6th Edition) diagnostic system for the selection of native and proficient English speakers at risk of reading difficulties. Journal of Psychoeducational Assessment. Advance online publication. doi:10.1177/0734282915589017 Google Scholar | SAGE Journals | ISI | |
|
Smolkowski, K., Cummings, K. D., Stryker, L. (in press). An introduction to the statistical evaluation of fluency measures with signal detection theory. In Cummings, K. D., Petscher, Y. (Eds.), Fluency metrics in education: Implications for test developers, researchers, and practitioners. New York, NY: Springer. Google Scholar | |
|
Tanner, W. P., Swets, J. A. (1954). A decision-making theory of signal detection. Psychological Review, 61, 401–409. Google Scholar | Crossref | Medline | ISI |
