![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
A framework for quantitative measurement development, validation, and revision that incorporates both qualitative and quantitative methods is introduced. It extends and adapts Adcock and Collier’s work, and thus, facilitates understanding of quantitative measurement development, validation, and revision as an integrated and cyclical set of procedures best achieved through mixed methods research. It also offers a systematic guide concerning how these procedures may be undertaken through detailing key “stages,” “levels,” and practical “tasks.” A case study illustrates how qualitative and quantitative methods may be mixed through the use of the proposed framework in the cross-cultural content- and construct-related validation and subsequent revision of a quantitative measure. The contribution of this article to mixed methods research literature is briefly discussed.
References
Section:
|
Adcock, R., Collier, D. (2001). Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review, 95, 529-546. Google Scholar | Crossref | ISI | |
|
Allen, J., Walsh, J. A. (2000). A construct-based approach to equivalence: Methodologies for cross-cultural/multicultural personality assessment research. In Dana, R. H. (Ed.), Handbook of cross-cultural and multicultural personality assessment (pp. 63-85). Mahwah, NJ: Lawrence Erlbaum. Google Scholar | |
|
American Educational Research Association . (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35, 33-40. Google Scholar | SAGE Journals | |
|
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education . (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Google Scholar | |
|
Beere, C. A. (1990). Gender roles: A handbook of tests and measures. New York, NY: Greenwood Press. Google Scholar | |
|
Ben-Porath, Y. S. (1990). Cross-cultural assessment of personality: The case for replicatory factor analysis. In Butcher, N., Spielberger, C. D. (Eds.), Advances in personality assessment (pp. 27-48). Hillsdale, NJ: Lawrence Erlbaum. Google Scholar | |
|
Bertrand, J. T., Brown, J. E., Ward, V. M. (1992). Techniques for analysing focus groups data. Evaluation Review, 16, 198-209. Google Scholar | SAGE Journals | ISI | |
|
Brannen, J. (1992). Combining qualitative and quantitative approaches: An overview. In Brannen, J. (Ed.), Mixing methods: Qualitative and quantitative research (pp. 3-37). Aldershot, England: Avebury. Google Scholar | |
|
Brannen, J. (2009). Mixed methods for novice researchers: Reflections and themes. International Journal of Multiple Research Approaches, 3, 8-12. Google Scholar | Crossref | |
|
Braun, V., Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77-101. Google Scholar | Crossref | |
|
Brislin, R. W. (2000). Back-translation. In Kazdin, A. E. (Ed.), Encyclopedia of psychology (pp. 359-360). Washington, DC: American Psychological Association. Google Scholar | Crossref | |
|
Campbell, D. T., Fiske, D. A. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105. Google Scholar | Crossref | Medline | ISI | |
|
Caracelli, V. J., Greene, J. C. (1993). Data analysis strategies for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 15, 195-207. Google Scholar | SAGE Journals | ISI | |
|
Coaley, K. (2010). An introduction to psychological assessment and psychometrics. London, England: SAGE. Google Scholar | |
|
Collins, K. M. T., Onwuegbuzie, A. J., Sutton, I. L. (2006). A model incorporating the rationale and purpose for conducting mixed methods research in special education and beyond. Learning Disabilities: A Contemporary Journal, 4, 67-100. Google Scholar | |
|
Creswell, J. W., Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: SAGE. Google Scholar | |
|
Denzin, N. K. (1970). The research act in sociology: The theoretical introduction to sociological methods. London, England: Butterworth. Google Scholar | |
|
Denzin, N. K. (1978). The logic of naturalistic inquiry. In Denzin, N. K. (Ed.), Sociological methods: A sourcebook (pp. 6-29). New York, NY: McGraw-Hill. Google Scholar | |
|
Floyd, F. J., Widaman, K. E. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286-299. Google Scholar | Crossref | ISI | |
|
Gergen, K. J. (1985). The social constructionist movement in modern psychology. American Psychologist, 40, 266-275. Google Scholar | Crossref | ISI | |
|
Gergen, K. J. (2005). An invitation to social construction. London, England: SAGE. Google Scholar | |
|
Greene, J. C., Caracelli, V. J., Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11, 255-274. Google Scholar | SAGE Journals | |
|
Griffin, C., Phoenix, A. (1994). The relationship between qualitative and quantitative research: Lessons from feminist psychology. Journal of Community & Applied Social Psychology, 4, 287-298. Google Scholar | Crossref | ISI | |
|
Hammersley, M. (1992). Deconstructing the qualitative-quantitative divide. In Brannen, J. (Ed.), Mixing methods: Qualitative and quantitative research (pp. 39-55). Aldershot, England: Avebury. Google Scholar | |
|
Hoffman, R. M. (2001). The measurement of masculinity and femininity: Historical perspective and implications for counselling. Journal of Counseling & Development, 79, 472-485. Google Scholar | Crossref | ISI | |
|
Hogan, T. P. (2007). Psychological testing: A practical introduction. New York, NY: John Wiley. Google Scholar | |
|
Hogan, T. P., Agnello, J. (2004). An empirical study of reporting practices concerning measurement validity. Educational and Psychological Measurement, 64, 802-812. Google Scholar | SAGE Journals | ISI | |
|
Jick, T. D. (1983). Mixing qualitative and quantitative methods: Triangulation in action. In Van Maanen, J. (Ed.), Qualitative methodology (pp. 135-148). Beverley Hills, CA: SAGE. Google Scholar | |
|
Johnson, R. B., Onwuegbuzie, A. J., Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133. Google Scholar | SAGE Journals | ISI | |
|
Kelle, U., Laurie, H. (1995). Computer use in qualitative research and issues of validity. In Kelle, U. (Ed.), Computer-aided qualitative data analysis (pp. 19-95). London, England: SAGE. Google Scholar | |
|
Levant, R. F., Hirsch, L. S., Celentano, E., Cozza, T. M., Hill, S., MacEachern, M., . . . Schnedeker, J. (1992). The male role: An investigation of contemporary norms. Journal of Mental Health Counseling, 14, 325-337. Google Scholar | |
|
Luyt, R. (2003). Rhetorical representations of masculinities in South Africa: Moving towards a material-discursive understanding of men. Journal of Community & Applied Social Psychology, 13, 46-69. Google Scholar | Crossref | ISI | |
|
Luyt, R. (2005). The Male Attitude Norms Inventory-II: A measure of masculinity ideology in South Africa. Men and Masculinities, 8, 208-229. Google Scholar | SAGE Journals | |
|
Luyt, R. (in press). Constructing hegemonic masculinities in South Africa: The discourse and rhetoric of heteronormativity. Gender and Language. Google Scholar | ISI | |
|
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749. Google Scholar | Crossref | ISI | |
|
Morgan, D. L. (2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48-76. Google Scholar | SAGE Journals | ISI | |
|
Morgan, G. A., Gliner, J. A., Harmon, R. J. (2001). Measurement validity. Journal of American Academy of Child & Adolescent Psychiatry, 40, 729-731. Google Scholar | Crossref | Medline | ISI | |
|
Nagy Hesse-Biber, C. (2010). Mixed methods research: Merging theory with practice. London, England: Guilford Press. Google Scholar | |
|
Nassar-McMillan, S. C., Borders, L. D. (2002). Use of focus groups in survey item development. The Qualitative Report, 7. Retrieved from http://www.nova.edu/ssss/QR/QR7-1/nassar.html Google Scholar | |
|
Neuman, W. L. (1997). Social research methods: Qualitative and quantitative approaches. Boston, MA: Allyn & Bacon. Google Scholar | |
|
O’Brien, K. (1993). Improving survey questionnaires through focus groups. In Morgan, D. L. (Ed.), Successful focus groups: Advancing the state of the art (pp. 105-117). Newbury Park, CA: SAGE. Google Scholar | Crossref | |
|
Onwuegbuzie, A. J., Bustamante, R. M., Nelson, J. A. (2010). Mixed research as a tool for developing quantitative instruments. Journal of Mixed Methods Research, 4(1), 56-78. Google Scholar | SAGE Journals | ISI | |
|
Pett, M. A., Lackey, N. R., Sullivan, J. J. (2003). Making sense of factor analysis: The use of factor analysis for instrument development in health care research. London, England: SAGE. Google Scholar | Crossref | |
|
Ponterotto, J. G., Grieger, I. (1999). Merging qualitative and quantitative perspectives in a research identity. In Kopala, M., Suzuki, L. A. (Eds.), Using qualitative methods in psychology (pp. 49-62). London, England: SAGE. Google Scholar | Crossref | |
|
Rust, J., Golombok, S. (2008). Modern psychometrics: The science of psychological assessment. London, England: Routledge. Google Scholar | |
|
Tashakkori, A., Teddlie, C. (1998). Introduction to mixed method and mixed model studies in the social and behavioural sciences. In Tashakkori, A., Teddlie, C. (Eds.), Mixed methodology: Combining qualitative and quantitative approaches (pp. 3-19). Thousand Oaks, CA: SAGE. Google Scholar | |
|
Tashakkori, A., Teddlie, C. (Eds.). (2003). Handbook of mixed methods in social & behavioral research. Thousand Oaks, CA: SAGE. Google Scholar | |
|
Teddlie, C., Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. London, England: SAGE. Google Scholar | |
|
Thompson, E. H., Pleck, J. H. (1995). Masculinity ideologies: A review of research instrumentation on men and masculinities. In Levant, R. F., Pollack, W. S. (Eds.), A new psychology of men (pp. 129-163). New York, NY: Basic Books. Google Scholar | |
|
Thompson, E. H., Pleck, J. H., Ferrera, D. L. (1992). Men and masculinities: Scales for masculinity ideology and masculinity-related constructs. Sex Roles, 27, 573-607. Google Scholar | Crossref | Medline | ISI | |
|
Tran, T. V. (2009). Developing cross-cultural measurement. Oxford, England: Oxford University Press. Google Scholar | Crossref | |
|
van de Vijver, F. J. R., Poortinga, Y. H. (2005). Conceptual and methodological issues in adapting tests. In Hambleton, R. K., Merenda, P. F., Spielberger, C. D. (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 39-63). Mahwah, NJ: Lawrence Erlbaum. Google Scholar | |
|
van de Vijver, F. J. R., Tanzer, N. K. (2004). Bias and equivalence in cross-cultural assessment: An overview. Revue Européenne de Psychologie Appliquée, 54, 119-135. Google Scholar | Crossref | |
|
Webb, E. J., Campbell, D. T., Schwartz, R. D., Sechrest, L. (1966). Unobtrusive measures: Nonreactive research in the social sciences. Chicago, IL: Rand McNally. Google Scholar | |
|
Wolff, B., Knodel, J., Sittitrai, W. (1993). Focus groups and surveys as complementary research methods. In Morgan, D. L. (Ed.), Successful focus groups: Advancing the state of the art (pp. 118-136). Thousand Oaks, CA: SAGE. Google Scholar | Crossref | |
|
Yun, J., Ulrich, D. A. (2002). Estimating measurement validity: A tutorial. Adapted Physical Activity Quarterly, 19, 32-47. Google Scholar | Crossref | Medline | ISI |
