![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
At present, it is unclear whether math curriculum-based measurement (M-CBM) procedures provide a dependable measure of student progress in math computation because support for its technical properties is based largely upon a body of correlational research. Recent investigations into the dependability of M-CBM scores have found that evaluating probe design procedures is necessary to estimate the contribution of probe-related factors to variance in M-CBM scores. To extend this research, the present study compared a set of commercially available probes (i.e., AIMSweb) with an experimental set that was constructed to enhance consistency in item content across alternative forms. Two probes were randomly selected from each set and administered to 43 students in fifth grade. Generalizability analyses indicated that the experimental probes generated scores with less error variance than those of the AIMSweb probes. Follow-up analyses indicated that fewer administrations of the experimental probes were necessary to attain adequate reliability for relative decisions (i.e., >.80) and high reliability for absolute decisions (i.e., >.90). Results support prior studies and suggest that error variance is associated with M-CBM probe content and development procedures.
References
Section:
|
Allinder, R. M., Fuchs, L. S., Fuchs, D., Hamlett, C. L. (1992). Effects of summer break on math and spelling performance as a function of grade level. The Elementary School Journal, 92, 451–460. Retrieved from http://dx.doi.org/10.1086/461701 Google Scholar | |
|
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education . (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association. Google Scholar | |
|
American Psychiatric Association . (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author. Google Scholar | |
|
Batsche, G. M., Curtis, M. J., Dorman, C., Castillo, J. M., Porter, L. J. (2007). The Florida problem-solving/response to intervention model: Implementing a statewide initiative. In Jimerson, S. R., Burns, M. K., VanDerHeyden, A. M. (Eds.), Handbook of response to intervention (pp. 378–395). New York, NY: Springer. Retrieved from http://dx.doi.org/10.1007/978-0-387-49053-3_28 Google Scholar | |
|
Betts, J. (2010). Expanding school psychology’s horizons for understanding and conducting research: Commentary on the special series on statistical analysis. Journal of School Psychology, 48, 1–4. Retrieved from http://dx.doi.org/10.1016/j.jsp.2009.11.001 Google Scholar | |
|
Betts, J., Pickart, M., Heistad, D. (2009). An investigation of the psychometric evidence of CBM-R passage equivalence: Utility of readability statistics and equating for alternate forms. Journal of School Psychology, 47, 1–17. Retrieved from http://dx.doi.org/10.1016/j.jsp.2008.09.001 Google Scholar | |
|
Brennan, R. L. (2001). Generalizability theory. New York, NY: Springer. Retrieved from http://dx.doi.org/10.1007/978-1-4757-3456-0 Google Scholar | |
|
Bricker, D., Yovanoff, P., Capt, B., Allen, D. (2003). Use of a curriculum-based measure to corroborate eligibility decisions. Journal of Early Intervention, 26, 20–30. Retrieved from http://dx.doi.org/10.1177/105381510302600102 Google Scholar | |
|
Briesch, A. M., Swaminathan, H., Welsh, M., Chafouleas, S. M. (2014). Generalizability theory: A practical guide to study design, implementation, and interpretation. Journal of School Psychology, 52, 13–35. Retrieved from http://dx.doi.org/10.1016/j.jsp.2013.11.008 Google Scholar | |
|
Burns, M. K. (2012). Assessment research and school psychology: Introduction to the special series. School Psychology Review, 41, 243–245. Google Scholar | ISI | |
|
Christ, T. J., Ardoin, S. P. (2009). Curriculum-based measurement of oral reading: Passage equivalence and probe-set development. Journal of School Psychology, 47, 55–75. Retrieved from http://dx.doi.org/10.1016/j.jsp.2008.09.004 Google Scholar | |
|
Christ, T. J., Johnson-Gros, K. N., Hintze, J. M. (2005). An examination of alternate assessment durations when assessing multiple-skill computational fluency: The generalizability and dependability of curriculum-based outcomes within the context of educational decisions. Psychology in the Schools, 42, 615–622. Retrieved from http://dx.doi.org/10.1002/pits.20107 Google Scholar | |
|
Christ, T. J., Scullin, S., Tolbize, A., Jiban, C. L. (2008). Implications of recent research: Curriculum-based measurement of math computation. Assessment for Effective Intervention, 33, 198–205. Retrieved from http://dx.doi.org/10.1177/1534508407313480 Google Scholar | |
|
Christ, T. J., Vining, O. (2006). Curriculum-based measurement procedures to develop multiple-skill math computation probes: Evaluation of random and stratified stimulus-set arrangements. School Psychology Review, 35, 387–400. Google Scholar | ISI | |
|
Cooney, J. B., Swanson, H. L., Ladd, S. F. (1988). Acquisition of mental multiplication skill: Evidence for the transition between counting and retrieval strategies. Cognition and Instruction, 5, 323–345. Retrieved from http://dx.doi.org/10.1207/s1532690xci0504_5 Google Scholar | |
|
Crick, J. E., Brennan, R. L. (1983). Manual for GENOVA: A generalized analysis of variance system. Iowa City, IA: The American College Testing Program. Google Scholar | |
|
Crocker, L., Algina, J. (1986). Introduction to classical and modern test theory. Orlando, FL: Holt, Rinehart, and Winston. Google Scholar | |
|
Deno, S. L. (2003). Developments in curriculum-based measurement. The Journal of Special Education, 37, 184–192. Retrieved from http://dx.doi.org/10.1177/00224669030370030801 Google Scholar | |
|
Deno, S. L., Mirkin, P. K. (1977). Data-based program modification: A manual. Reston, VA: Council for Exceptional Children. Google Scholar | |
|
Dirks, E., Spyer, G., van Lieshout, E. C., de Sonneville, L. (2008). Prevalence of combined reading and arithmetic disabilities. Journal of Learning Disabilities, 41, 460–473. Retrieved from http://dx.doi.org/10.1177/0022219408321128 Google Scholar | |
|
Downing, S. M. (2006). Twelve steps for effective test development. In Downing, S. M., Haladyna, T. M. (Eds.), Handbook of test development (pp. 3–25). Mahwah, NJ: Lawrence Erlbaum. Google Scholar | |
|
Foegen, A., Jiban, C., Deno, S. L. (2007). Progress monitoring measures in math: A review of the literature. Journal of Special Education, 41, 121–139. Retrieved from http://dx.doi.org/10.1177/00224669070410020101 Google Scholar | |
|
Foegen, A., Olson, J. R., Impecoven-Lind, L. (2008). Developing progress monitoring measures for secondary math: An illustration in algebra. Assessment for Effective Intervention, 33, 240–249. Retrieved from http://dx.doi.org/10.1177/1534508407313489 Google Scholar | |
|
Fuchs, L. S. (2003). Assessing intervention responsiveness: Conceptual and technical issues. Learning Disabilities Research & Practice, 18, 172–186. Retrieved from http://dx.doi.org/10.1111/1540-5826.00073 Google Scholar | |
|
Fuchs, L. S. (2004). The past, present, and future of curriculum based measurement research. School Psychology Review, 33, 188–192. Google Scholar | ISI | |
|
Fuchs, L. S., Deno, S. L. (1991). Paradigmatic distinctions between instructionally relevant measurement models. Exceptional Children, 57, 488–500. Google Scholar | SAGE Journals | ISI | |
|
Fuchs, L. S., Fuchs, D. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53, 199–208. Google Scholar | SAGE Journals | ISI | |
|
Fuchs, L. S., Fuchs, D., Compton, D. L., Bryant, J. D., Hamlett, C. L., Seethaler, P. M. (2007). Math screening and progress monitoring at first grade: Implications for responsiveness to intervention. Exceptional Children, 73, 311–330. Retrieved from http://dx.doi.org/10.1177/001440290707300303 Google Scholar | |
|
Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L., Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22, 27–48. Google Scholar | ISI | |
|
Fuchs, L. S., Fuchs, D., Phillips, N., Hamlett, C., Karns, K. (1995). Acquisition and transfer effects of classwide peer-assisted learning strategies in math for students with varying learning histories. School Psychology Review, 24, 604–620. Google Scholar | ISI | |
|
Fuchs, L. S., Fuchs, D., Powell, S. R., Seethaler, P. M., Cirino, P. T., Fletcher, J. M. (2008). Intensive intervention for students with math disabilities: Seven principles of effective practice. Learning Disability Quarterly, 31, 79–92. Google Scholar | SAGE Journals | ISI | |
|
Fuchs, L. S., Hamlett, C. L., Fuchs, D. (1998). Monitoring basic skills progress: Basic math computation. Austin, TX: Pro-Ed. Google Scholar | |
|
Geary, D. C. (2004). Math and learning disabilities. Journal of Learning Disabilities, 37, 4–15. Retrieved from http://dx.doi.org/10.1177/00222194040370010201 Google Scholar | |
|
Geary, D. C., Hoard, M. K., Nugent, L., Bailey, D. H. (2012). Mathematical cognition deficits in children with learning disabilities and persistent low achievement: A five-year prospective study. Journal of Educational Psychology, 104, 206–233. Retrieved from http://dx.doi.org/10.1037/a0025398 Google Scholar | |
|
Gersten, R., Beckmann, S., Clarke, B., Foegen, A., March, L., Star, J. R., Witzel, B. (2009). Assisting students struggling with mathematics: Response to Intervention (RtI) for elementary and middle schools (Practice Guide Report). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Google Scholar | |
|
Halfon, N., Houtrow, A., Larson, K., Newacheck, P. W. (2012). The changing landscape of disability in childhood. The Future of Children, 22, 13–42. Retrieved from http://dx.doi.org/10.1353/foc.2012.0004 Google Scholar | |
|
Hauser, M. D., Chomsky, N., Fitch, W. T. (2002). The faculty of language: What is it, who has it, and how did it evolve? Science, 298, 1569–1579. Retrieved from http://dx.doi.org/10.1126/science.298.5598.1569 Google Scholar | |
|
Hintze, J. M., Christ, T. J., Keller, L. A. (2002). The generalizability of CBM survey-level math assessments: Just how many samples do we need? School Psychology Review, 31, 514–528. Google Scholar | ISI | |
|
Hosp, J. L. (2012). Using assessment data to make decisions about teaching and learning. In Harris, K. R., Graham, S., Urdan, T. (Eds.), APA educational psychology handbook: Application to learning and teaching (Vol. 3, pp. 87–109). Washington, DC: American Psychological Association. Google Scholar | Crossref | |
|
Keller-Margulis, M., Shapiro, E. S., Hintze, J. M. (2008). Long-term diagnostic accuracy of curriculum-based measures in reading and math. School Psychology Review, 37, 374–390. Google Scholar | ISI | |
|
Lei, P., Smith, M., Suen, H. (2007). The use of generalizability theory to estimate data reliability in single-subject observational research. Psychology in the Schools, 44, 433–439. Retrieved from http://dx.doi.org/10.1002/pits.20235 Google Scholar | |
|
Lembke, E., Stecker, P. (2007). Curriculum-based measurement in math: An evidence-based formative assessment procedure. Portsmouth, NH: RMC Research Corporation, Center on Instruction. Google Scholar | |
|
Linn, R. L. (2006). The standards for educational and psychological testing: Guidance in test development. In Downing, S. M., Haladyna, T. M. (Eds.), Handbook of test development (pp. 27–38). Mahwah, NJ: Lawrence Erlbaum. Google Scholar | |
|
Marston, D. B. (1989). A curriculum-based measurement approach to assessing academic performance: What it is and why do it. In Shinn, M. R. (Ed.), Curriculum-based measurement: Assessing special children (pp. 18–78). New York, NY: Guilford Press. Google Scholar | |
|
Mazzocco, M. M. M. (2007). Defining and differentiating mathematical learning disabilities and difficulties. In Berch, D. B., Mazzocco, M. M. M. (Eds.), Why is math so hard for some children? The nature and origins of mathematical learning difficulties and disabilities (pp. 7–27). Baltimore, MD: Paul H. Brookes. Google Scholar | |
|
National Governors Association Center for Best Practices & Council of Chief State School Officers . (2010). Common core state standards for math. Washington, DC: Author. Google Scholar | |
|
Seethaler, P. M., Fuchs, L. S. (2011). Using curriculum-based measurement to monitor kindergarteners’ math development. Assessment for Effective Intervention, 36, 219–229. Retrieved from http://dx.doi.org/10.1177/1534508411413566 Google Scholar | |
|
Shalev, R. S. (2007). Prevalence of developmental dyscalculia. In Berch, D. B., Mazzocco, M. M. M. (Eds.), Why is math so hard for some children? The nature and origins of mathematical learning difficulties and disabilities (pp. 49–60). Baltimore, MD: Paul H. Brookes. Google Scholar | |
|
Shapiro, E. S. (2008). From research to practice: Promoting academic competence for underserved students. School Psychology Review, 37, 46–51. Google Scholar | ISI | |
|
Shapiro, E. S., Edwards, L., Zigmond, N. (2005). Progress monitoring of math among students with learning disabilities. Assessment for Effective Intervention, 30, 15–32. Retrieved from http://dx.doi.org/10.1177/073724770503000203 Google Scholar | |
|
Shinn, M. R. (1989). Curriculum-based measurement: Assessing special children. New York, NY: Guilford Press. Google Scholar | |
|
Shinn, M. R. (2004). Administration and scoring of math computation curriculum-based measurement (M-CBM) and math fact probes for use with AIMSweb. Available from www.aimsweb.com Google Scholar | |
|
Smith, P. L. (1981). Gaining accuracy in generalizability theory: Using multiple designs. Journal of Educational Measurement, 18, 147–154. Retrieved from http://dx.doi.org/10.1111/j.1745-3984.1981.tb00849.x Google Scholar | |
|
Stecker, P. M., Fuchs, L. S. (2000). Effecting superior achievement using curriculum-based measurement: The importance of individual progress monitoring. Learning Disabilities Research & Practice, 15, 128–134. Retrieved from http://dx.doi.org/10.1207/SLDRP1503_2 Google Scholar | |
|
Thurber, R. S., Shinn, M. R., Smolkowski, K. (2002). What is measured in math tests? Construct validity of curriculum-based math measures. School Psychology Review, 31, 498–513. Google Scholar | ISI | |
|
VanDerHeyden, A. M. (2011). Technical adequacy of response to intervention decisions. Exceptional Children, 77, 335–350. Retrieved from http://dx.doi.org/10.1177/001440291107700305 Google Scholar | |
|
VanDerHeyden, A. M., Burns, M. K. (2005). Using curriculum-based assessment and curriculum-based measurement to guide elementary math instruction: Effect on individual and group accountability scores. Assessment for Effective Intervention, 30, 15–31. Retrieved from http://dx.doi.org/10.1177/073724770503000302 Google Scholar | |
|
Vanderwood, M. L., Linklater, D., Healy, K. (2008). Predictive accuracy of nonsense word fluency for English language learners. School Psychology Review, 37, 5. Google Scholar | ISI | |
|
Volpe, R. J., Briesch, A. M. (2012). An investigation of the generalizability and dependability of single item and multiple item Direct Behavior Rating scales. School Psychology Review, 41, 246–261. Google Scholar | ISI | |
|
Ysseldyke, J. E., Vanderwood, M. L., Shriner, J. (1997). Changes over the past decade in special education referral to placement probability: An incredibly reliable practice. Assessment for Effective Intervention, 23, 193–201. Google Scholar |
