![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Even as the importance of replication research has become more widely understood, the field of gifted education is almost completely devoid of replication studies. An area in which replication is a particular problem is in student identification research, since instrument validity is a necessary prerequisite for any sound psychometric decision. To begin to address this issue, our study sought to replicate the internal validity structure of three teacher rating instruments. The goal was to determine whether data gathered using these instruments fit their published internal validity structures. Results indicated all three instruments failed to meet traditional fit criteria, but to varying degrees, and that further replication or instrument revision are needed before these instruments can be used with confidence.
Keywords gifted education, assessment, validity, replication, identification
References
Section:
|
Bollen, K. A., Stein, R. A. (1992). Bootstrapping goodness-of-fit measures in structural equation models. Sociological Methods & Research, 21, 205-229. doi:10.1177/0049124192021002004 Google Scholar | SAGE Journals | ISI | |
|
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press. Google Scholar | |
|
Callahan, C. M., Moon, T. R., Oh, S. (2013). Status of elementary gifted programs: 2013. Charlottesville, VA: National Research Center on the Gifted and Talented. Google Scholar | |
|
Dai, D. Y., Swanson, J. A., Cheng, H. (2011). State of research on giftedness and gifted education: A survey of empirical studies published during 1998-2010 (April). Gifted Child Quarterly, 55, 126-138. doi:10.1177/0016986210397831 Google Scholar | SAGE Journals | ISI | |
|
Fan, X., Thompson, B. (1998, September). Bootstrap estimation of sample statistic bias in structural equation modeling. Paper presented at the annual meeting of the American Education Research Association, San Diego, CA. Google Scholar | |
|
Finney, S. J., DiStefano, C. (2006). Non-normal and categorical data in structural equation modeling. In Hancock, G. R., Mueller, R. O. (Eds.), Structural equation modeling: A second course (pp. 269-314). Greenwich, CT: Information Age Publishing. Google Scholar | |
|
French, B. F., Finch, W. H. (2006). Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modeling, 13, 378-402. doi:10.1207/s15328007sem1303_3 Google Scholar | Crossref | ISI | |
|
Gagné, F. (1994). Are teachers really poor talent detectors? Comments on Pegnato and Birch’s (1959) study of the effectiveness and efficacy of various identification techniques. Gifted Child Quarterly, 38, 124-126. doi:10.1177/001698629403800305 Google Scholar | SAGE Journals | |
|
Geisinger, K. F., Spies, R. A., Carlson, J. F., Plake, B. S. (Eds.). (2007). The seventeenth mental measurements yearbook. Lincoln, NE: The Buros Institute of Mental Measurements. Google Scholar | |
|
Gentry, M., Pereira, N. P., Peters, S. J., McIntosh, J., Fugate, C. M. (2015). HOPE Teacher Rating Scale: Involving teachers in equitable identification of gifted and talented students in K-12 [Administration manual]. Waco, TX: Prufrock Press. Google Scholar | |
|
Gentry, M., Peters, S. J., Pereira, N. P., McIntosh, J., Fugate, C. M. (2015). HOPE Teacher Rating Scale: Involving teachers in equitable identification of gifted and talented students in K-12 [Measurement instrument]. Waco, TX: Prufrock Press. Google Scholar | |
|
Hu, L., Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. doi:10.1080/10705519909540118 Google Scholar | Crossref | ISI | |
|
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, 696-701. Google Scholar | Crossref | ISI | |
|
John, L. K., Loewenstein, G., Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524-532. doi:10.1177/0956797611430953 Google Scholar | SAGE Journals | ISI | |
|
Kenny, D. (2015). Measuring model fit. Retrieved from http://davidakenny.net/cm/fit.htm Google Scholar | |
|
Lohman, D. F. (2006). Identifying academically talented minority students (Research Monograph RM05216). Storrs: The National Research Center on the Gifted and Talented, University of Connecticut. Google Scholar | |
|
Makel, M. C. (2014). The empirical march: Making science better at self-correction. Psychology of Aesthetics, Creativity, and the Arts, 8, 2-7. doi:10.1037/a0035803 Google Scholar | Crossref | ISI | |
|
Makel, M. C., Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43, 304-316. doi:10.3102/0013189X14545513 Google Scholar | SAGE Journals | ISI | |
|
Makel, M. C., Plucker, J. A. (2015). An introduction to replication research in gifted education: Shiny and new is not the same as useful. Gifted Child Quarterly, 59, 157-164. Google Scholar | SAGE Journals | ISI | |
|
Makel, M. C., Plucker, J. A., Hegarty, B. (2012). Replication in psychology research: How often do they really occur? Perspectives on Psychological Science, 7, 537-542. doi: 10.1177/1745691612460688 Google Scholar | SAGE Journals | |
|
Matthews, M. S. (2007). [Review of the Scales for Identifying Gifted Students by G. R. Ryser & K. McConnell]. In Geisinger, K. F., Spies, R. A., Calrson, J. F., Plake, B. S. (Eds.), The seventeenth mental measurements yearbook (pp. 729-731). Lincoln, NE: The Buros Institute of Mental Measurements. Google Scholar | |
|
McBee, M. T., Matthews, M. S. (2014). Welcoming quality in non-significance and replication work, but moving beyond the p-value. Journal of Advanced Academics, 25, 73-87. doi:10.1177/1932202X14532177 Google Scholar | SAGE Journals | |
|
McBee, M. T., Peters, S. J., Miller, E. M. (2016). The impact of the nomination stage on gifted program identification: A comprehensive psychometric analysis. Gifted Child Quarterly, 60, 258-278. doi:10.1177/0016986216656256 Google Scholar | SAGE Journals | ISI | |
|
McBee, M. T., Peters, S. J., Waterman, C. (2014). Combining scores in multiple-criteria assessment systems: The impact of combination rule. Gifted Child Quarterly, 58, 69-89. doi:10.1177/0016986213513794 Google Scholar | SAGE Journals | ISI | |
|
National Association for Gifted Children . (2015). State of the states in gifted education: 2014-2015. Washington, DC: Author. Google Scholar | |
|
Pegnato, C. W., Birch, J. W. (1959). Locating gifted children in junior high schools: Comparison of methods. Exceptional Children, 25, 300-304. Google Scholar | SAGE Journals | ISI | |
|
Pereira, N. (2011). Finding and understanding talent among elementary English Language Learners: Beyond project HOPE (Doctoral dissertation). Purdue University, West Lafayette, IN. Available from ProQuest Dissertations and Theses database. (UMI No. 3481123) Google Scholar | |
|
Peters, S. J., Engerrand, K. G. (2016). Equity and excellence: Proactive efforts in the identification of underrepresented students for gifted and talented services. Gifted Child Quarterly, 60, 159-171. doi:10.1177/0016986216643165. Google Scholar | SAGE Journals | |
|
Peters, S. J., Gentry, M. (2010). Multi-group construct validity evidence of the HOPE Scale: Instrumentation to identify low-income elementary students for gifted programs. Gifted Child Quarterly, 54, 298-313. doi:10.1177/0016986210378332 Google Scholar | SAGE Journals | |
|
Peters, S. J., Gentry, M. (2012). Additional validity evidence and across-group equivalency of the HOPE Teacher Rating Scale. Gifted Child Quarterly, 57, 85-100. doi:10.1177/0016986212469253 Google Scholar | SAGE Journals | |
|
Pfeiffer, S. I., Jarosewich, T. (2003). Gifted Rating Scales. San Antonio, TX: Pearson Education. Google Scholar | |
|
Pfeiffer, S. I., Jarosewich, T. (2007). The Gifted Rating Scales–School Form: An analysis of the standardization sample based on age, gender, race, and diagnostic efficiency. Gifted Child Quarterly, 51, 39-50. doi:10.1177/0016986206296658 Google Scholar | SAGE Journals | ISI | |
|
Renzulli, J. S., Smith, L. H., White, A. J., Callahan, C. M., Hartman, R. K., Westberg, K. L., . . . Reen, R. E. (2010). Scales for rating the behavioral characteristics of superior students (3rd ed.). Waco, TX: Prufrock Press. Google Scholar | |
|
Ryser, G. R., McConnell, K. (2004). Scales for identifying gifted students. Waco, TX: Prufrock Press. Google Scholar | |
|
Sandilos, L. (2017). [Review of the HOPE Teacher Rating Scale by M. Gentry, S. J. Peters, N. Pereira, J. S. McIntosh, & C. M. Fugate]. In Calrson, J. F., Geisinger, K. F., Jonson, J. L. (Eds.), The twentieth mental measurements yearbook. Retrieved from http://marketplace.unl.edu/buros/ Google Scholar | |
|
Sullivan, J. R. (2017). [Review of the HOPE Teacher Rating Scale by M. Gentry, S. J. Peters, N. Pereira, J. S. McIntosh, & C. M. Fugate]. In Calrson, J. F., Geisinger, K. F., Jonson, J. L. (Eds.), The twentieth mental measurements yearbook. Retrieved from http://marketplace.unl.edu/buros/ Google Scholar | |
|
Thompson, M. S., Green, S. B. (2006). Evaluating between-group differences in latent variable means. Structural equation modeling: A second course (pp. 119-169). Greenwich, CT: Information Age Publishing. Google Scholar | |
|
U.S. Department of Education . (1993). National excellence: The case for developing America’s talent. Washington, DC: U.S. Government Printing Office. Google Scholar | |
|
Ward, S. (2005). [Review of the Gifted Rating Scales by S. I. Pfeiffer & T. Jarosewich]. In Spies, R. A., Calrson, J. F., Plake, B. S. (Eds.), The sixteenth mental measurements yearbook (pp. 1311-1312). Lincoln, NE: The Buros Institute of Mental Measurements. Google Scholar | |
|
Ward, S. (2007). [Review of the Scales for Identifying Gifted Students by G. R. Ryser & K. McConnell]. In Geisinger, K. F., Spies, R. A., Calrson, J. F., Plake, B. S. (Eds.), The seventeenth mental measurements yearbook (pp. 731-734). Retrieved from http://marketplace.unl.edu/buros/ Google Scholar | |
|
Wyner, J. S., Bridgeland, J. M., Diiulio, J. J. (2009). Achievement trap (Rev. ed.). Washington, DC: The Jack Kent Cooke Foundation. Google Scholar | |
|
Yoon, S., Gentry, M. (2009). Racial and ethnic representation in gifted programs: Current status of and implications for gifted Asian American students. Gifted Child Quarterly, 53, 121-136. doi:10.1177/0016986208330564 Google Scholar | SAGE Journals | ISI |
About the Authors
Scott J. Peters is an associate professor of Educational Foundations at the University of Wisconsin – Whitewater where he teaches courses on measurement and assessment, research methodology, and gifted education. His research work focuses on educational assessment, gifted and talented student identification underrepresentation, and educational policy. His work has appeared in journals such as Gifted Child Quarterly, the Journal of Advanced Academics, Ed Week, Ed Leadership, Gifted Child Today, and Gifted and Talented International.
Nielsen Pereira (PhD) is an assistant professor of Gifted, Creative, and Talented Studies at Purdue University. His research interests include conceptual, contextual, and measurement issues in the identification of gifted and talented populations, design and assessment of learning in varied gifted and talented education contexts, and understanding gifted and talented student experiences in talent development programs in and out of school. He is a regular presenter in national and international conferences on educational research, gifted education, and STEM education.
