![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
State-level testing programs continue to grow, and the challenge of validation does not wane. Although more than a decade has passed since the 1999 Joint Standards for Educational and Psychological Testing set out a call for the organization of validity evidence into validity arguments, practical examples of such arguments are not readily available for the research community. The purpose of this article is to demonstrate a validity argument for a large-scale alternate assessment, using one state program as a model. Specifically, the authors review the documented purposes of the assessment program and the assumptions related to teaching and learning subsumed within those purpose statements. Potential sources of evidence to support or refute those assumptions are also presented.
References
Section:
|
Almond, P., Bechard, S. (2005). In-depth look at students who take alternate assessments: What do we know now? Retrieved from http://www.measuredprogress.org/assessments/inclusive/articlespapers.aspx Google Scholar | |
|
American Psychological Association, American Educational Research Association, & National Council on Measurement in Education . (1974). Standards for educational and psychological tests. Washington, DC: American Psychological Association. Google Scholar | |
|
American Psychological Association, American Educational Research Association, & National Council on Measurement in Education . (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association. Google Scholar | |
|
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education . (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Google Scholar | |
|
Behuniak, P., Amenta, J. (2009). Assessing all students in Connecticut. In Schafer, W. M., Lissitz, R. (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 157–170). Baltimore, MD: Brookes. Google Scholar | |
|
Browder, D. M., Fallin, K., Davis, S., Karvonen, M. (2003). A consideration of what may influence student outcomes on alternate assessment. Education and Training in Developmental Disabilities, 38, 255–270. Google Scholar | |
|
Browder, D. M., Flowers, C., Ahlgrim-Delzell, L., Karvonen, M., Spooner, F., Algozzine, R. (2004). The alignment of alternate assessment content to academic and functional curricula. Journal of Special Education, 37, 211–223. Google Scholar | SAGE Journals | ISI | |
|
Browder, D. M., Spooner, F., Algozzine, B., Ahlgrim-Delzell, L., Flowers, C., Karvonen, M. (2003). What we know and need to know about alternate assessment. Exceptional Children, 70, 45–62. Google Scholar | SAGE Journals | ISI | |
|
Browder, D., Wakeman, S., Flowers, C. (2009). Which came first: The curriculum or the assessment? In Schafer, W. M., Lissitz, R. (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 329–334). Baltimore, MD: Brookes. Google Scholar | |
|
Chapelle, C. A., Enright, M. K., Jamieson, J. M. (2010). Does an argument-based approach to validity make a difference? Educational Measurement: Issues and Practice, 29(1), 3–13. Google Scholar | Crossref | |
|
Cizek, G. J., Rosenberg, S., Koons, H. (2008). Sources of validity evidence for educational and psychological tests. Educational and Psychological Measurement, 68, 397–412. Google Scholar | SAGE Journals | ISI | |
|
Connecticut State Department of Education . (2006). CMT/CAPT Skills Checklist Second Generation: Technical manual. Hartford: Connecticut State Department of Education. Google Scholar | |
|
Cronbach, L. J. (1988). Five perspectives on validity argument. In Wainer, H. (Ed.), Test validity (pp. 3–17). Hillsdale, NJ: Lawrence Erlbaum. Google Scholar | |
|
Elliott, S. N., Compton, E., Roach, A. T. (2007). Concurrent and predictive validity evidence for Idaho Alternate Assessment scores of students with and without significant disabilities. Educational Measurement: Issues and Practice, 26(2), 30–42. Google Scholar | Crossref | |
|
Elliott, S. N., Roach, A. T. (2007). Alternate assessments of students with significant disabilities: Alternative approaches, common technical challenges. Applied Measurement in Education, 20, 301–333. Google Scholar | Crossref | ISI | |
|
Ericsson, K. A., Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Rev. ed.). Cambridge, MA: MIT Press. Google Scholar | |
|
Flowers, C., Browder, D., Ahlgrim-Delzell, L. (2006). An analysis of three states’ alignment between language arts and mathematics standards and alternate assessments. Exceptional Children, 72, 201–215. Google Scholar | SAGE Journals | ISI | |
|
Goldstein, J., Behuniak, P. (in press). Assessing students with significant cognitive disabilities on academic content: The teachers’ perspective. Journal of Special Education. Google Scholar | |
|
Haertel, E. (1999). Validity arguments for high-stakes testing: In search of the evidence. Educational Measurement: Issues and Practice, 18(4), 5–9. Google Scholar | Crossref | |
|
Haladyna, T. M. (2006). Roles and importance of validity studies in test development. In Downing, S. M., Haladyna, T. M. (Eds.), Handbook of test development (pp. 739–758). Mahwah,NJ: Lawrence Erlbaum. Google Scholar | |
|
Hogan, T. P., Agnello, J. (2004). An empirical study of reporting practices concerning measurement validity. Educational and Psychological Measurement, 64, 802–812. Google Scholar | SAGE Journals | ISI | |
|
Individuals with Disabilities Education Improvement Act, Pub. L. No. 108-446, 118 Stat. 2647 (2004). Google Scholar | |
|
Johnson, E., Arnold, N. (2004). Validating an alternate assessment. Remedial and Special Education, 25, 266–275. Google Scholar | SAGE Journals | |
|
Kampfer, S., Horvath, L., Kleinert, H., Kearns, J. (2001). Teachers’ perceptions of one state’s alternate assessment portfolio program: Implications for practice and teacher preparation. Exceptional Children, 67, 361–374. Google Scholar | SAGE Journals | |
|
Kane, M. T. (2006). Validation. In Brennan, R. L. (Ed.), Educational measurement (4th ed., pp. 17–64). Washington, DC: National Council on Measurement in Education and American Council on Education. Google Scholar | |
|
Kane, M. (2008). Terminology, emphasis, and utility in validation. Educational Researcher, 37, 76–82. Google Scholar | SAGE Journals | ISI | |
|
Karvonen, M., Flowers, C. P., Browder, D. M., Wakeman, S., Algozzine, B. (2006). Case study of the influences on alternate assessment outcomes for students with disabilities. Education and Training in Developmental Disabilities, 41, 95–110. Google Scholar | |
|
Kearns, J. F., Towles-Reeves, E., Kleinert, H. L., Kleinert, J., Thomas, M. (in press). Characteristics of and implications for students participating in alternate assessments based on alternate academic achievement standards. Journal of Special Education. Google Scholar | |
|
Kleinert, H. L., Kearns, J. F., Kennedy, S. (1997). Accountability for all students: Kentucky’s alternate portfolio assessment for students with moderate and severe cognitive disabilities. Journal of the Association for Persons with Severe Handicaps, 22, 88–101. Google Scholar | SAGE Journals | |
|
Linn, R. (2002). Validation of the uses and interpretations of results of state assessment and accountability systems. In Tindal, G., Haladyna, T. (Eds.), Large-scale assessment programs for all students: Validity, technical adequacy, and implementation (pp. 27–48). Mahwah, NJ: Lawrence Erlbaum. Google Scholar | |
|
Linn, R. (2009). The concept of validity in the context of NCLB. In Lissitz, R. (Ed.), The concept of validity: Revisions, new directions and applications (pp. 195–212). Charlotte, NC: Information Age. Google Scholar | |
|
Lissitz, R. W., Samuelsen, K. (2007). A suggested change in terminology and emphasis regarding validity and education. Educational Researcher, 36, 437–448. Google Scholar | SAGE Journals | |
|
Marion, S., Perie, M. (2009). An introduction to validity arguments for alternative assessments. In Schafer, W. M., Lissitz, R. (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 329–334). Baltimore, MD: Brookes. Google Scholar | |
|
Mehrens, W. (2002). Consequences of assessment: What is the evidence? In Tindal, G., Haladyna, T. (Eds.), Large-scale assessment programs for all students: Validity, technical adequacy, and implementation. Mahwah, NJ: Lawrence Earlbaum. Google Scholar | |
|
Messick, S. (1989). Validity. In Linn, R. L. (Ed.), Educational measurement (3rd ed., pp. 13–103). Washington, DC: American Council on Education and National Council on Measurement in Education. Google Scholar | |
|
Moss, P. A. (2007). Reconstructing validity. Educational Researcher, 36, 470–476. Google Scholar | SAGE Journals | |
|
No Child Left Behind Act, Pub. L. No. 107-110, 115 Stat. 1425 (2002). Google Scholar | |
|
Quenemoen, R. (2008). A brief history of alternate assessments based on alternate achievement standards (Synthesis Report 68). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Google Scholar | |
|
Quenemoen, R. (2009). Toward a straighter road. In Schafer, W. M., Lissitz, R. (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 359–362). Baltimore, MD: Brookes. Google Scholar | |
|
Reckase, M. (1998). Consequential validity from the test developer’s perspective. Educational Measurement: Issues and Practice, 17(2), 13-16. Google Scholar | Crossref | |
|
Rigney, S. (2009). Public policy and the development of alternate assessments for students with cognitive disabilities. In Schafer, W. M., Lissitz, R. (Eds.), Alternate assessments based on alternate achievement standards: Policy, practice, and potential (pp. 41–60). Baltimore, MD: Brookes. Google Scholar | |
|
Ryan, K. (2002). Assessment validation in the context of high stakes assessments. Educational Measurement: Issues and Practice, 2(1), 7–15. Google Scholar | Crossref | |
|
Schafer, W. D., Wang, J., Wang, V. (2009). Validity in action: State assessment validity evidence for compliance with NCLB.In Lissitz, R. (Ed.), The concept of validity: Revisions, new directions and applications (pp. 173–193). Charlotte, NC: Information Age. Google Scholar | |
|
Sireci, S. G. (2007). On validity theory and test validation. Educational Researcher, 36, 477–481. Google Scholar | SAGE Journals | |
|
Sireci, S. G. (2009). Packing and unpacking sources of validity evidence: History repeats itself again. In Lissitz, R. (Ed.), The concept of validity: Revisions, new directions and applications (pp. 19–37). Charlotte, NC: Information Age. Google Scholar | |
|
Standards for Educational and Psychological Testing . (n.d.). Timeline. Retrieved from http://teststandards.org/timeline.htm Google Scholar | |
|
Tindal, G., McDonald, M., Tedesco, M., Glasgow, A., Almond, P., Crawford, L., Hollenbeck, K. (2003). Alternate assessments in reading and math: Development and validation for students with significant disabilities. Exceptional Children, 49, 481–494. Google Scholar | SAGE Journals | |
|
Towles-Reeves, E., Kearns, J., Kleinert, H., Kleinert, J. (2009). An analysis of the learning characteristics of students taking alternate assessments based on alternate achievement standards. Journal of Special Education, 42, 241–254. Google Scholar | SAGE Journals | ISI | |
|
U.S. Department of Education . (2003). Title I–Improving the academic achievement of the disadvantaged: Final rule, 68 Fed. Reg. 236 (December 9, 2003). Washington, DC: U.S. Department of Education, Office of Elementary and SecondaryEducation. Google Scholar | |
|
U.S. Department of Education . (2005, August). Alternate achievement standards for students with the most significant cognitive disabilities: Non-regulatory guidance. Washington, DC: Author. Google Scholar | |
|
U.S. Department of Education . (2009). State and local implementation of the No Child Left Behind Act, volume V: Implementation of the 1 percent rule and 2 percent interim policy options. Washington, DC: Office of Planning, Evaluation, and Policy Development. Retrieved from http://www.ed.gov/rschstat/eval/disadv/nclb-disab/nclb-disab.pdf Google Scholar |
