![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
This study examined the predictive validity and classification accuracy of two commonly used universal screening measures relative to a statewide achievement test. Results indicated that second-grade performance on oral reading fluency and the Measures of Academic Progress (MAP), together with special education status, explained 68% of the variance in third-grade standardized test performance (N = 399). Six methods of dichotomous risk classification were also explored to predict whether students would pass or fail the state test. A random sample of half of the participants was used to establish locally appropriate risk cut-scores, holding sensitivity constant at .95; the cut-scores were then cross-validated on the remaining participants. Cross-validation reflected sensitivity of above .90 for all methods, with MAP Language Usage and two multi-screener methods demonstrating the strongest overall technical adequacy for risk classification. The limitations and implications of these findings are discussed.
References
Section:
|
Akobeng, A. K. (2007). Understanding diagnostic tests 2: Likelihood ratios, pre- and post-test probabilities and their use in clinical practice. Acta Paediatrica, 96, 487–491. doi:10.1111/j.1651-2227.2006.00179.x Google Scholar | Crossref | Medline | ISI | |
|
Andren, K. J. (2010). An analysis of the concurrent and predictive validity of curriculum-based measures (CBM), the Measures of Academic Progress (MAP), and the New England Common Assessment Program (NECAP) for reading (Unpublished doctoral dissertation). University of Southern Maine, Portland. Google Scholar | |
|
Ardoin, S. P., Christ, T. J. (2008). Evaluating curriculum-based measurement slope estimates using data from triannual universal screenings. School Psychology Review, 37, 109–125. Google Scholar | ISI | |
|
Ball, C. R., Christ, T. J. (2012). Supporting valid decision making: Uses and misuses of assessment data within the context of RTI. Psychology in the Schools, 49, 231–244. doi:10.1002/pits.21592 Google Scholar | Crossref | ISI | |
|
Ball, C. R., Finch, W. H., Gettinger, M., & K-3 Reading and Behavior Intervention Project . (2014). Classroom-level effects on the reading and behavior of at-risk kindergartners. Preventing School Failure: Alternative Education for Children and Youth, 58, 80–89. Google Scholar | |
|
Bishop, A. G., League, M. B. (2006). Identifying a multivariate screening model to predict reading difficulties at the onset of kindergarten: A longitudinal analysis. Learning Disability Quarterly, 29, 235–252. Google Scholar | SAGE Journals | ISI | |
|
Clemens, N. H., Shapiro, E. S., Thoemmes, F. (2011). Improving the efficacy of first grade reading screening: An investigation of word identification fluency with other early literacy indicators. School Psychology Quarterly, 26, 231–244. doi:10.1037/a0025173 Google Scholar | Crossref | ISI | |
|
Compton, D. L., Fuchs, D., Fuchs, L. S., Bryant, J. D. (2006). Selecting at-risk readers in first grade for intervention: A two-year longitudinal study of decision rules and procedures. Journal of Educational Psychology, 98, 394–409. doi:10.1037/0022-0663.98.2.394 Google Scholar | Crossref | ISI | |
|
Fuchs, L. S. (2004). The past, present, and future of curriculum-based measurement research. School Psychology Review, 33, 188–192. Google Scholar | ISI | |
|
Fuchs, L. S., Deno, S. L. (1991). Paradigmatic distinctions between instructionally relevant measurement models. Exceptional Children, 57, 488–500. Google Scholar | SAGE Journals | ISI | |
|
Glover, T. A., Albers, C. A. (2007). Considerations for evaluating universal screening assessments. Journal of School Psychology, 45, 117–135. doi:10.1016/j.jsp.2006.05.005 Google Scholar | Crossref | ISI | |
|
Goffreda, C. T., DiPerna, J. C. (2010). An empirical review of psychometric evidence for the Dynamic Indicators of Basic Early Literacy Skills. School Psychology Review, 39, 463–483. Google Scholar | ISI | |
|
Good, R. H., Kaminski, R. A. (Eds.). (2002). Dynamic Indicators of Basic Early Literacy Skills (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement. Available from http://dibels.uoregon.edu Google Scholar | |
|
Good, R. H., Kaminski, R. A., Smith, S., Bratten, J. (2001). Technical adequacy of second grade DIBELS Oral Reading Fluency passages (Technical Report No. 8). Eugene: University of Oregon. Google Scholar | |
|
Hintze, J. M., Silberglitt, B. (2005). A longitudinal analysis of the diagnostic accuracy and predictive validity of R-CBM and high-stakes testing. School Psychology Review, 34, 372–386. Google Scholar | ISI | |
|
Jenkins, J. R., Hudson, R. F., Johnson, E. S. (2007). Screening for at-risk readers in a response-to-intervention framework. School Psychology Review, 36, 582–600. Google Scholar | ISI | |
|
Johnson, E. S., Jenkins, J. R., Petscher, Y. (2010). Improving the accuracy of a direct route screening process. Assessment for Effective Intervention, 35, 131–142. doi:10.1177/1534508409348375 Google Scholar | SAGE Journals | |
|
Johnson, E. S., Jenkins, J. R., Petscher, Y., Catts, H. W. (2009). How can we improve the accuracy of screening instruments? Learning Disabilities Research & Practice, 24, 174–185. doi:10.1111/j.1540-5826.2009.00291.x Google Scholar | Crossref | |
|
Kilgus, S. P., Methe, S. A., Maggin, D. M., Tomasula, J. L. (2014). Curriculum-based measurement of oral reading (R-CBM): A diagnostic test accuracy meta-analysis of evidence supporting use in universal screening. Journal of School Psychology, 52, 377–405. doi:10.1016/j.jsp.2014.06.002 Google Scholar | Crossref | Medline | ISI | |
|
Kingsbury Center at Northwest Evaluation Association . (2010). Wisconsin linking study: A study of the alignment of the NWEA RIT scale with the Wisconsin Knowledge and Concepts Examination. Lake Oswego, OR: Author. Google Scholar | |
|
MacDonald, H. H., Sullivan, A. L., Watkins, M. W. (2013). Multivariate screening model for later word reading achievement: Predictive utility of prereading skills and cognitive ability. Journal of Applied School Psychology, 29, 52–71. doi:10.1080/15377903.2013.751476 Google Scholar | Crossref | |
|
Maziarz, A. L. (2010). The exploration of demographics and computer adaptive testing in predicting performance on state-mandated reading assessments (Unpublished doctoral dissertation). Indiana University of Pennsylvania, Indiana. Google Scholar | |
|
Mellard, D. F., McKnight, M., Woods, K. (2009). Response to intervention screening and progress monitoring practices in 41 local schools. Learning Disabilities Research & Practice, 24, 186–195. Google Scholar | Crossref | |
|
Merino, K., Beckman, T. O. (2010). Using reading curriculum-based measurements as predictors for the Measures of Academic Progress (MAP) standardized test in Nebraska. International Journal of Psychology: A Biopsychosocial Approach, 6, 85–98. Google Scholar | |
|
Moyer, E. L., Galindo, J. L., Dodd, B. G. (2012). Balancing flexible constraints and measurement precision in computerized adaptive testing. Educational and Psychological Measurement, 72, 629–648. doi:10.1177/0013164411431838 Google Scholar | SAGE Journals | ISI | |
|
National Association of School Psychologists . (2009). School psychologists’ involvement in assessment (Position statement). Bethesda, MD: Author. Google Scholar | |
|
National Center on Response to Intervention. (2013, January). Screening briefs series—Brief #1: Classification accuracy. Washington, DC: U.S. Department of Education, Office of Special Education Programs, National Center on Response to Intervention. Retrieved from http://www.rti4success.org/sites/default/files/RTI%20Screening%20Brief1-Classification%20Accuracy.pdf Google Scholar | |
|
Northwest Evaluation Association . (2009). Technical manual for measures of academic progress and measures of academic progress for primary grades. Lake Oswego, OR: Author. Google Scholar | |
|
Northwest Evaluation Association . (2011). 2011 normative data. Lake Oswego, OR: Author. Google Scholar | |
|
Petscher, Y., Kim, Y., Foorman, B. R. (2011). The importance of predictive power in early screening assessments: Implications for placement in the response to intervention framework. Assessment for Effective Intervention, 36, 158–166. doi:10.1177/1534508410396698 Google Scholar | SAGE Journals | |
|
Reschly, A. L., Busch, T. W., Betts, J., Deno, S. L., Long, J. D. (2009). Curriculum-based measurement oral reading as an indicator of reading achievement: A meta-analysis of the correlational evidence. Journal of School Psychology, 47, 427–469. doi:10.1016/j.jsp.2009.07.001 Google Scholar | Crossref | Medline | ISI | |
|
VanDerHeyden, A. M. (2011). Technical adequacy of RtI decisions. Exceptional Children, 77, 335–350. Google Scholar | SAGE Journals | ISI | |
|
VanDerHeyden, A. M. (2013). Universal screening may not be for everyone: Using a threshold model as a smarter way to determine risk. School Psychology Review, 42, 402–414. Google Scholar | ISI | |
|
Wang, S., McCall, M., Jiao, H., Harris, G. (2013). Construct validity and measurement invariance of computerized adaptive testing: Application to Measures of Academic Progress (MAP) using confirmatory factor analysis. Journal of Educational and Developmental Psychology, 3, 88–100. doi:10.5539/jedp.v3n1p88 Google Scholar | Crossref | |
|
Wayman, M. M., Wallace, T., Wiley, H. I., Ticha, R., Espin, C. A. (2007). Literature synthesis on curriculum-based measurement in reading. The Journal of Special Education, 41, 85–120. doi:10.1177/00224669070410020401 Google Scholar | SAGE Journals | ISI | |
|
Wisconsin Department of Public Instruction . (2005). Wisconsin student assessment system criterion-referenced test framework for the WKCE statewide assessment. Madison: Author. Google Scholar | |
|
Zellmer, M. B., Frontier, A., Pheifer, D. (2006). What are NCLB’s instructional costs? Educational Leadership, 64, 43–46. Google Scholar | ISI |
