![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Progress monitoring has been adopted as an integral part of multi-tiered support systems. Oral reading fluency (ORF) is the most established assessment for progress-monitoring purposes. To generate valid trend lines or slopes, ORF passages must be of equivalent difficulty. Recently, however, evidence indicates that ORF passages are not equivalent, potentially hindering our ability to generate valid student trend lines for decision making. This study examines passage and order effects on the estimation of ORF scores using a set of second-grade passages. A single group with counterbalancing design was employed to randomly assign 156 second-grade students to three different orders of passages. Scores from the passages were examined using growth curve modeling and empirical Bayes estimates. Results indicate that passage effects were substantial, but order effects were small but significant. The impact of passage and order effects on research design, equating methods, and measure development is considered.
References
Section:
|
Albano, A. D., Rodriguez, M. C. (2012). Statistical equating with measures of oral reading fluency. Journal of School Psychology, 50, 43–59. Google Scholar | Crossref | Medline | ISI | |
|
Alonzo, J., Tindal, G. (2007). The development of fifth-grade passage reading fluency measures for use in a progress monitoring assessment system (Technical Report No. 40). Eugene, OR: Behavioral Research & Teaching. Retrieved from http://www.brtprojects.org/publications/dl/54 Google Scholar | |
|
Anderson, R. C., Davison, A. (1988). Conceptual and empirical bases of readability formulas. In Davisson, A., Green, G. M. (Eds.), Linguistic complexity and text comprehension: Readability issues reconsidered (pp. 23–53). Hillsdale, NJ: Lawrence Erlbaum. Google Scholar | |
|
Ardoin, S. P., Christ, T. J. (2009). Curriculum-based measurement of oral reading: Standard errors associated with progress monitoring outcomes from DIBELS, AimsWeb, and an experimental passage set. School Psychology Review, 38, 266–283. Google Scholar | ISI | |
|
Ardoin, S. P., Suldo, S. M., Witt, J., Aldrich, S., McDonald, E. (2005). Accuracy of readability estimates’ predictions of CBM performance. School Psychology Quarterly, 20, 1–22. doi:10.1521/scpq.20.1.1.64193 Google Scholar | Crossref | ISI | |
|
Ardoin, S. P., Williams, J. C., Christ, T. J., Klubnik, C., Wellborn, C. (2010). Examining readability estimates’ predictions of students’ oral reading rate: Spache, Lexile, and Forcast. School Psychology Review, 39, 277–285. Google Scholar | ISI | |
|
Bates, D., Maechler, M., Bolker, B., Walker, S. (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67, 1-48. doi:10.18637/jss.v067.i01 Google Scholar | Crossref | ISI | |
|
Bailin, A., Grafstein, A. (2001). The linguistic assumptions underlying readability formulae: A critique. Language & Communication, 21, 285–301. Google Scholar | Crossref | ISI | |
|
Baker, S. K., Fien, H., Baker, D. L. (2010). Robust reading instruction in the early grades: Conceptual and practical issues in the integration and evaluation of Tier 1 and Tier 2 instructional supports. Focus on Exceptional Children, 42, 1–20. Google Scholar | Crossref | |
|
Baker, S. K., Smolkowski, K., Katz, R., Fien, H., Seeley, J., Kame’enui, E., Beck, C. (2008). Reading fluency as a predictor of reading proficiency in low-performing high poverty schools. School Psychology Review, 37, 18–37. Google Scholar | ISI | |
|
Betts, J., Pickart, M., Heistad, D. (2009). An investigation of the psychometric evidence of CBM-R passage equivalence: Utility of readability statistics and equating for alternate forms. Journal of School Psychology, 47, 1–17. doi:10.1016/j.jsp.2008.09.001 Google Scholar | Crossref | ISI | |
|
Bolt, D. M., Ysseldyke, J., Patterson, M. J. (2010). Students, teachers, and schools as sources of variability, integrity, and sustainability in implementing progress monitoring. School Psychology Review, 39, 612–630. Google Scholar | |
|
Bruce, B., Rubin, A. (1988). Readability formulas: Matching tool and task. In Davisson, A., Green, G. M. (Eds.), Linguistic complexity and text comprehension: Readability issues reconsidered (pp. 5–22). Hillsdale, NJ: Lawrence Erlbaum. Google Scholar | |
|
Buck, J., Torgesen, J. (2003). The relationship between performance on a measure of oral reading fluency and performance on the Florida Comprehensive Assessment Test (FCRR Technical Report #1). Tallahassee: Florida Center for Reading Research. Google Scholar | |
|
Christ, T. J., Ardoin, S. P. (2009). Curriculum-based measurement of oral reading: Passage equivalence and probe-set development. Journal of School Psychology, 47, 55–75. doi:10.1016/j.jsp.2008.09.004 Google Scholar | Crossref | ISI | |
|
Christ, T. J., Silberglitt, B. (2007). Curriculum-based measurement of oral reading fluency: The standard error of measurement. School Psychology Review, 36, 130–146. Google Scholar | ISI | |
|
Christ, T. J., Silberglitt, B., Yeo, S., Cormier, D. (2010). Curriculum-based measurement of oral reading: An evaluation of growth rates and seasonal effects among students served in general and special education. School Psychology Review, 39, 447–462. doi:10.1177/1534508412456417 Google Scholar | SAGE Journals | ISI | |
|
Christ, T. J., Zopluoglu, C., Long, J., Monaghen, B. (2012). Curriculum-based measurement of oral reading: Quality of progress monitoring outcomes. Exceptional Children, 78, 356–373. Google Scholar | SAGE Journals | ISI | |
|
Cummings, K. D., Park, Y., Schaper, H. (2013). Equating DIBELS Next Oral Reading Fluency passages. Assessment for Effective Intervention, 38, 91–104. Google Scholar | SAGE Journals | |
|
Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219–232. doi:10.1177/004005998702000109 Google Scholar | SAGE Journals | ISI | |
|
Deno, S. L., Reschly, A. L., Lembke, E. S., Magnusson, D., Callender, S. A., Windram, H., Stachel, N. (2009). Developing a School-Wide Progress-Monitoring System. Psychology in the Schools, 46, 44–55. Google Scholar | Crossref | ISI | |
|
Dorans, N. J. (1990). Equating methods and sampling designs. Applied Measurement in Education, 3, 3–17. doi:10.1207/s15324818ame0301_2 Google Scholar | Crossref | |
|
Francis, D. J., Santi, K. L., Barr, C., Fletcher, J. M., Varisco, A., Foorman, B. R. (2008). Form effects on the estimation of students’ oral reading fluency using DIBELS. Journal of School Psychology, 46, 315–342. doi:10.1016/j.jsp.2007.06.003 Google Scholar | Crossref | Medline | ISI | |
|
Fuchs, L. S., Deno, S. L. (1992). Effects of curriculum within curriculum-based measurement. Exceptional Children, 58, 232–243. doi:10.1177/001440299105800306 Google Scholar | SAGE Journals | ISI | |
|
Fuchs, L. S., Fuchs, D., Deno, S. L. (1982). Reliability and validity of curriculum-based informal reading inventories. Reading Research Quarterly, 18, 6–26. doi:10.2307/747536 Google Scholar | Crossref | ISI | |
|
Fuchs, L. S., Fuchs, D., Hamlett, C. L., Ferguson, C. (1992). Effects of expert system consultation within curriculum-based measurement using a reading maze task. Exceptional Children, 58, 436–450. Google Scholar | SAGE Journals | ISI | |
|
Good, R. H., Kaminski, R. (2002a). DIBELS oral reading fluency passages for first through third grades (Technical Report No. 10). Eugene: University of Oregon. Google Scholar | |
|
Good, R. H., Kaminski, R. (2002b). DIBELS 6th edition administration and scoring guide. Eugene: University of Oregon. Available from http://dibels.uoregon.edu/ Google Scholar | |
|
Good, R. H., Kaminski, R. (2002c). Dynamic indicators of basic early literacy skills (6th ed.). Eugene, OR: Institute for the Development of Education Achievement. Google Scholar | |
|
Griffiths, A.-J., VanDerHeyden, A. M., Skokut, M., Lilles, E. (2009). Progress monitoring in oral reading fluency within the context of RTI. School Psychology Quarterly, 24, 13–23. doi:10.1037/a0015435 Google Scholar | Crossref | ISI | |
|
Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., Wayman, J. (2009). Using student achievement data to support instructional decision making (NCEE 2009-4067). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Google Scholar | |
|
Hosp, M. K., Hosp, J. L., Howell, K. W. (2007). The ABCs of CBM: A practical guide to curriculum-based measurement. New York, NY: Guilford. Google Scholar | |
|
Howe, K. B., Shinn, M. M. (2002). Standard Reading Assessment Passages (RAPs) for use in general outcome measurement: A manual describing development and technical features. Eden Prairie, MN: Edformation. Google Scholar | |
|
Kekahio, W., Baker, M. (2013). Five steps for structuring data-informed conversations and action in education (REL 2013-001). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Pacific. Retrieved from http://ies.ed.gov/ncee/edlabs Google Scholar | |
|
Kolen, M. J., Brennan, R. L. (2014). Test equating, scaling, and linking: Methods and practices (3rd ed.). New York, NY: Springer Verlag. Google Scholar | Crossref | |
|
Lembke, E. S., McMaster, K. L., Stecker, P. M. (2010). The prevention science of reading research within a response to intervention model. Psychology in the Schools, 47, 22–35. doi:10.1002/pits.20449 Google Scholar | Crossref | ISI | |
|
National Association of State Directors of Special Education . (2006). Response to intervention: Policy considerations and implementation. Alexandria, VA: Author. Google Scholar | |
|
Nese, J., Biancarosa, G., Anderson, D., Lai, C.-F., Alonzo, J., Tindal, G. (2013). Within-year oral reading fluency with CBM: A comparison of models. Reading and Writing, 25, 887–915. doi:10.1007/s11145-011-9304-0 Google Scholar | Crossref | |
|
Paris, S. G. (2005). Reinterpreting the development of reading skills. Reading Research Quarterly, 40, 184–202. doi:10.1598/RRQ.40.2.3 Google Scholar | Crossref | ISI | |
|
Petscher, Y., Cummings, K. D., Biancarosa, G., Fien, H. (2013). Advanced (measurement) applications of curriculum-based measurement in reading. Assessment for Effective Intervention, 38, 71–75. doi:10.1177/1534508412461434 Google Scholar | SAGE Journals | |
|
Poncy, B. C., Skinner, C. H., Axtell, P. K. (2005). An investigation of the reliability and standard error of measurement of words read correctly per minute using curriculum-based measurement. Journal of Psychoeducational Assessment, 23, 326–338. doi:10.1177/073428290502300403 Google Scholar | SAGE Journals | ISI | |
|
Powell-Smith, K. A., Bradley-Klug, K. L. (2001). Another look at the “C” in CBM: Does it really matter if curriculum-based measurement reading probes are curriculum-based? Psychology in the Schools, 38, 299–312. doi:10.1002/pits.1020 Google Scholar | Crossref | |
|
Preacher, K. J., Wichman, A. L., MacCallum, R. C., Briggs, N. E. (2008). Latent growth curve modeling. Los Angeles, CA: SAGE. Google Scholar | Crossref | |
|
R Core Team (2017). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria. URL https://www.R-project.org/. Google Scholar | |
|
Shinn, M. R. (1989). Curriculum-based measurement: Assessing special children. New York, NY: Guilford. Google Scholar | |
|
Silberglitt, B., Hintze, J. (2005). Formative assessment using CBM-R cut scores to track progress toward success on state-mandated achievement tests: A comparison of methods. Journal of Psychoeducational Assessment, 23, 304–325. doi:10.1177/073428290502300402 Google Scholar | SAGE Journals | ISI | |
|
Stoolmiller, M., Biancarosa, G., Fien, H. (2013). Measurement properties of DIBELS oral reading fluency in Grade 2: Implications for equating studies. Assessment for Effective Intervention, 38, 76–90. doi:10.1177/1534508412456729 Google Scholar | SAGE Journals | |
|
Zakaluk, B. L., Samuels, S. J. (Eds.). (1988). Readability: Its past, present, and future. Newark, DE: International Reading Association. Google Scholar |
