![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Social validity is the extent to which procedures, goals, and outcomes are acceptable and important to end users and is not commonly addressed during assessment development. This article pilots a process of triangulating multiple methods to evaluate the social validity of a self-report assessment. These procedures were piloted using the Child Occupational Self-Assessment (COSA), a self-report of everyday activities for children with disabilities. This study collected COSA responses from 502 children with disabilities and conducted observations and interviews with 5 additional children with cerebral palsy. Data were aligned with social validity benchmark statements and integrated using data displays. Social validity was evaluated by examining congruencies and discrepancies in the results. Implications and the legitimacy of design decisions are discussed.
|
American Occupational Therapy Association. ( 2002). Occupational therapy practice framework: Domain and process . American Journal of Occupational Therapy, 56, 609-639. Google Scholar | Medline | ISI | |
|
Betz, C.L. , Redcay, G. , & Tan, S. ( 2003). Self-reported health care self-care needs of transition age youth: A pilot study. Issues in Contemporary Pediatric Nursing, 26, 159-181. Google Scholar | Crossref | Medline | |
|
Bryman, A. ( 2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1(1), 8-22. Google Scholar | SAGE Journals | ISI | |
|
Cavet, J. , & Sloper, P. ( 2004). Participation of disabled children in individual decisions about their lives and in public decisions about service development. Children & Society, 18, 278-290. Google Scholar | Crossref | |
|
Creswell, J.W. , Plano Clark, V.L. , Gutmann, M.L. , & Hanson, W.E. ( 2003). Advanced mixed methods research designs. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-240). Thousand Oaks, CA: SAGE. Google Scholar | |
|
Dellinger, A.B. , & Leech, N.L. ( 2007). Toward a unified validation framework in mixed methods research. Journal of Mixed Methods Research, 1(4), 309-332. Google Scholar | SAGE Journals | ISI | |
|
Fasoli, L. ( 2003). Reading photographs of young children: looking at practices . Contemporary Issues in Early Childhood, 4, 32-47. Google Scholar | SAGE Journals | |
|
Fawcett, S.B. ( 1991). Social validity: A note on methodology. Journal of Applied Behavior Analysis, 24, 235-239. Google Scholar | Crossref | Medline | ISI | |
|
Foster, S.L. , & Mash, E.J. ( 1999). Assessing social validity in clinical treatment research issues and procedures. Journal of Consulting and Clinical Psychology, 67, 308-319. Google Scholar | Crossref | Medline | ISI | |
|
Gan, C. , Campbell, K.A. , Snider, A. , Cohen, S. , & Hubbard, J. ( 2008). Giving Youth a Voice (GYV): A measure of youth’s perceptions of the client-centredness of rehabilitation services. Canadian Journal of Occupational Therapy, 75, 96-104. Google Scholar | SAGE Journals | |
|
Greene, J. ( 2008). Is mixed methods social inquiry a distinctive methodology ? Journal of Mixed Methods Research, 2(1), 7-22. Google Scholar | SAGE Journals | ISI | |
|
Hemmingsson, H. , Kottorp, A. , & Bernspång, B. (2004). Validity of the school setting interview: an assessment of the student-environment fit. Scandinavian Journal of Occupational Therapy, 11, 171-178. Google Scholar | Crossref | |
|
Hitchcock, J.H. , Nastasi, B.K. , Dai, D.Y. , Newman, J. , Jayasena, A. , Bernstein-Moore, R. , . . . Varjas, K. ( 2005). Illustrating a mixed-method approach for validating culturally specific constructs. Journal of School Psychology, 43, 259-278. Google Scholar | Crossref | ISI | |
|
Jang, E.E. , McDougall, D.E. , Pollon, D. , Herbert, M. , & Russell, P. ( 2008). Integrative mixed methods data analytic strategies in research on school success in challenging circumstances. Journal of Mixed Methods Research, 2(3), 221-247. Google Scholar | SAGE Journals | ISI | |
|
Johnson, R.B. , & Onwuegbuzie, A.J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher , 33(7), 14-26. Google Scholar | SAGE Journals | |
|
Keller, J. , Kafkes, A. , Basu, S. , Federico, J. , & Kielhofner, G. (2005). The Child Occupational Self Assessment (COSA) Version 2.1. Chicago, IL: MOHO Clearinghouse, Department of Occupational Therapy, College of Applied Health Sciences, University of Illinois at Chicago. Google Scholar | |
|
King, G. , Law, M. , King, S. , Hanna, S. , Kertoy, M. , Rosenbaum, P. , & Young, N. ( 2004). Children’s Assessment of Participation and Enjoyment (CAPE) and Preferences for Activities of Children (PAC). San Antonio, TX: Harcourt Assessment. Google Scholar | |
|
Kramer, J. , & Hammel, J. (IN PRESS). "I do lots of things:" Children with cerebral palsy’s competence for everyday activities. International Journal of Disability, Development, and Education. Google Scholar | ISI | |
|
Kramer, J. , Smith, E.V., Jr. , & Kielhofner, G. (2009). Rating scale use by children with disabilities on a self report of everyday activities. Archives of Physical Medicine and Rehabilitation, 90, 2047-2053. Google Scholar | Crossref | Medline | ISI | |
|
Lee, Y.J. , & Greene, J. ( 2007). The predictive validity of an ESL placement test: A mixed methods approach. Journal of Mixed Methods Research, 1(4), 366-389. Google Scholar | SAGE Journals | ISI | |
|
Masters, G.N. ( 1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174. Google Scholar | Crossref | ISI | |
|
McConney, A. , Rudd, A. , & Ayres, R. ( 2002). Getting to the bottom line: A method for synthesizing findings within mixed-methods program evaluations. American Journal of Evaluation, 23, 121-140. Google Scholar | SAGE Journals | ISI | |
|
Messick, S. ( 1995). Validity of Psychological Assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749. Google Scholar | Crossref | ISI | |
|
Messick, S. ( 1998). Test validity: A matter of consequence. Social Indicators Research, 45, 35-44. Google Scholar | Crossref | ISI | |
|
Miles, M. B., & Huberman, A. B. (Eds.). (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: SAGE. Google Scholar | |
|
Morgan, D.L. ( 2007). Paradigms lost and pragmatism regained: methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48-76. Google Scholar | SAGE Journals | ISI | |
|
Morse, J.M. ( 1998). Designing funded qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), Collecting and interpreting qualitative materials (pp. 56-85). Thousand Oaks, CA: SAGE. Google Scholar | |
|
Myford, C.M. , & Mislevy, R.J. ( 1995). Monitoring and improving a portfolio assessment system . Princeton, NJ: Center for Performance Assessment. Google Scholar | |
|
Onwuegbuzie, A.J. , & Collins, K. ( 2007). A typology of mixed methods sampling designs in social science research. The Qualitative Report, 12, 281-316. Google Scholar | |
|
Onwuegbuzie, A.J. , & Johnson, R.B. ( 2006). The validity issue in mixed research. Research in the Schools, 13, 48-63. Google Scholar | |
|
Patton, M.Q. ( 2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks, CA: SAGE. Google Scholar | |
|
Rebok, G. , Riley, A. , Forrest, C. , Starfield, B. , Green, B. , Robertson, J. , & Tambor, E. ( 2001). Elementary school-aged children’s reports of their health: A cognitive interviewing study. Quality of Life Research , 10, 59-70. Google Scholar | Crossref | Medline | ISI | |
|
Rost, J. ( 1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement , 14, 271-282. Google Scholar | SAGE Journals | ISI | |
|
Smith, E.V., Jr. , Conrad, K.M. , Chang, K. , & Piazza, J. ( 2002). An introduction to Rasch measurement for scale development and person assessment. Journal of Nursing Measurement, 10, 189-206. Google Scholar | Crossref | Medline | |
|
Strauss, A. , & Corbin, J. ( 1998). Grounded theory methodology: An overview. In N. K. Denzin & Y. S. Lincoln (Eds.), Strategies of qualitative inquiry (pp. 158-183). Thousand Oaks, CA: SAGE. Google Scholar | |
|
Sturgess, J. , Rodger, S. , & Ozanne, A. ( 2002). A review of the use of self-report assessment with young children. British Journal of Occupational Therapy, 65, 108-116. Google Scholar | SAGE Journals | |
|
Taylor, S.J. , & Bogdan, R. ( 1998). Introduction to qualitative research methods: A guidebook and resource (3rd ed.). New York, NY: Wiley. Google Scholar | |
|
Teddlie, C. , & Yu, F. ( 2007). Mixed methods sampling: A typology with examples. Journal of Mixed Methods Research, 1(1), 77-100. Google Scholar | ISI | |
|
Turner, C. ( 2003). Are you listening? What disabled children and young people in Wales think about the services they use. Cardiff, Wales : Barnardos, Children First, NCH. Google Scholar | |
|
Valnes, B. , Hare, R. , Block, J. , Branch, C. , & Broshears, G. ( 2003). Future directions agenda for youth with disabilities, 2003. Portland, OR: National Youth Leadership Network. Google Scholar | |
|
Wagner-Menghin, M.M. ( 2006). The Mixed-Rasch Model: An example for analyzing the meaning of response latencies in a personality questionnaire. Journal of Applied Measurement, 7(2), 225-237. Google Scholar | Medline | |
|
Wolf, M.M. ( 1978). Social validity: The case for subjective measurement, or, how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203-214. Google Scholar | Crossref | Medline | ISI | |
|
World Health Organization. (2001). International classification of functioning, disability, and health. Geneva, Switzerland: Author. Google Scholar | |
|
Wright, B.D. , & Linacre, M. ( 1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 70, 857-860. Google Scholar | |
|
Wright, B.D. , & Stone, M.H. ( 1979). Best test design. Chicago, IL: MESA Press. Google Scholar | |
|
Young, B. , Rice, H. , Dixon-Woods, M. , Colver, A.F. , & Parkinson, K.N. (2007). A qualitative study of the health-related quality of life of disabled children. Developmental Medicine & Child Neurology, 49, 660-665. Google Scholar | Crossref | Medline | ISI |
