![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Reliability and validity are integral concepts in assessment design. Test speededness, the influence of time constraints on test taker performance, is often an overlooked threat to reliability and validity, especially in classroom-based testing. The purpose of this study is to evaluate the degree of test speededness of classroom-based assessments that have been developed and tested for a curricular reading intervention study with third-grade gifted students. The degree of speededness for the assessments was calculated using a mixture Rasch model. The results indicate that a large proportion of students were influenced by test speededness for both posttests in both treatment groups. The implications of these results are discussed.
Keywords speededness, validity, reliability
References
Section:
|
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education . (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association. Google Scholar | |
|
Baxter, B. (1941). An experimental analysis of speed and level in an intelligence test. Journal of Educational Psychology, 32, 285-296. Google Scholar | Crossref | |
|
Bejar, I. I. (1985). Test speededness under number-right scoring: An analysis of the Test of English As a Foreign Language (Report No. ETS-RR-85-11). Princeton, NJ: Educational Testing Service. Google Scholar | |
|
Bolt, D., Cohen, A., Wollack, J. (2002). Item parameter estimation under conditions of test speededness: Application of a mixture Rasch model with ordinal constraints. Journal of Educational Measurement, 39, 331-348. Google Scholar | Crossref | ISI | |
|
Camilli, G., Shepard, L. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage. Google Scholar | |
|
Crocker, L., Algina, J. (1986). Factors that affect reliability coefficients. In Crocker, L., Algina, J. (Eds.), Introduction to classical and modern test theory (pp. 143-146). Fort Worth, TX: Harcourt Brace Jovanovich. Google Scholar | |
|
Davidson, W. M., Carroll, J. B. (1945). Speed and level components in time limit scores: A factor analysis. Educational and Psychological Measurement, 5, 411-427. Google Scholar | SAGE Journals | ISI | |
|
Douglas, J., Kim, H. R., Habing, B., Gao, F. (1998). Investigating local dependence with conditional covariance functions. Journal of Educational and Behavioral Statistics, 23, 129-151. Google Scholar | SAGE Journals | ISI | |
|
Embretson, S., Reise, S. (2009). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum. Google Scholar | |
|
Evans, F. R., Reilly, R. R. (1972). A study of speededness as a source of test bias. Journal of Educational Measurement, 9, 123-131. Google Scholar | Crossref | ISI | |
|
Furr, R. M., Bacharach, V. R. (2008). Psychometrics: An introduction. Thousand Oaks, CA: Sage. Google Scholar | |
|
Green, S. B., Hershberger, S. L. (2000). Correlated errors in true score models and their effect on coefficient alpha. Structural Equation Modeling, 7, 251-270. Google Scholar | Crossref | ISI | |
|
Gulliksen, H. (1950). Theory of mental tests. Hillsdale, NJ: Lawrence Erlbaum. Google Scholar | Crossref | |
|
Hambleton, R. K., Swaminathan, H. (1985). Item response theory: Principles and applications. Boston, MA: Kluwer-Nijhoff. Google Scholar | Crossref | |
|
Henderson, W. (2004). The LSAT, law school exams, and meritocracy: The surprising and undertheorized role of test-taking speed. Texas Law Review, 82, 975-1052. Google Scholar | ISI | |
|
Kaplan, S. A. (2001). Layering differentiated curriculum for the gifted and talented. In Karnes, F., Bean, S. (Eds.), Methods and materials for teaching the gifted. Waco, TX: Prurock Press. Google Scholar | |
|
Kurpius, S., Stafford, M. (2006). Testing and measurement: A user-friendly guide. Thousand Oaks, CA: Sage. Google Scholar | |
|
Lawrence, I. M. (1993). The effect of test speededness on subgroup performance (Report no. ETS-RR-93–49). Princeton, NJ: Educational Testing Service. Google Scholar | |
|
Lord, F. M., Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley. Google Scholar | |
|
Lu, Y., Sireci, S. (2007). Validity issues in test speededness. Educational Measurement: Issues and Practice, 26(4), 29-37. Google Scholar | Crossref | |
|
Meyer, J. P. (2010). Understanding measurement: Reliability. New York, NY: Oxford University Press. Google Scholar | Crossref | |
|
Meyer, J. P. (2011). jMetrik (Version 2.0.1) [Software]. Retrieved from http://itemanalysis.com/jmetrik-download.php Google Scholar | |
|
Mroch, A. A., Bolt, D. M., Wollack, J. A. (2005, April). A new multi-class mixture Rasch model for test speededness. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Quebec. Google Scholar | |
|
No Child Left Behind (NCLB) Act of 2001 , Pub. L. No. 107–110. § 115, Stat. 1425. (2002). Google Scholar | |
|
Oshima, T. C. (1994). The effect of speededness on parameter estimation in item response theory. Journal of Educational Measurement, 21, 200-219. Google Scholar | Crossref | ISI | |
|
Renzulli, J. S., Reis, S. M. (1985). The schoolwide enrichment model: A comprehensive plan for educational excellence. Mansfield Center, CT: Creative Learning Press. Google Scholar | |
|
Schnipke, D. L. (1996). How contaminated by guessing are item-parameter estimates and what can be done about it? Paper presented at the annual meeting of the National Council on Measurement in Education, New York, NY. Google Scholar | |
|
Spiegelhalter, D., Thomas, A., Best, N., Lunn, D. (2007). Openbugs: User manual (Version 3.2.1). Retrieved from http://www.openbugs.info/w/FrontPage Google Scholar | |
|
Thorndike, R., Thorndike-Christ, T. (2009). Measurement and evaluation in psychology and education. Englewood Cliffs, NJ: Prentice Hall. Google Scholar | |
|
Tomlinson, C. A. (2001). How to differentiate instruction in mixed-ability classrooms (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development. Google Scholar | |
|
van der Linden, W. J. (2011). Setting time limits on tests. Applied Psychological Measurement, 35, 183-199. Google Scholar | SAGE Journals | ISI | |
|
van der Linden, W. J., Scrams, D. J., Schnipke, D. L. (1999). Using response-time constraints to control for differential speededness in computerized adaptive testing. Applied Psychological Measurement, 23, 195-210. Google Scholar | SAGE Journals | ISI | |
|
Wollack, J. A., Cohen, A. S., Wells, C. S. (2003). A method for maintaining scale stability in the presence of test speededness. Journal of Educational Measurement, 40, 307-330. Google Scholar | Crossref | ISI | |
|
Zenisky, A., Hambleton, R., Sireci, S. (2002). Identification and evaluation of local item dependencies in the medical college admissions test. Journal of Educational Measurement, 39, 291-309. Google Scholar | Crossref | ISI |
