![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Many psychometric measures yield data that are compatible with (a) an essentially unidimensional factor analysis solution and (b) a correlated-factor solution. Deciding which of these structures is the most appropriate and useful is of considerable importance, and various procedures have been proposed to help in this decision. The only fully developed procedures available to date, however, are internal, and they use only the information contained in the item scores. In contrast, this article proposes an external auxiliary procedure in which primary factor scores and general factor scores are related to relevant external variables. Our proposal consists of two groups of procedures. The procedures in the first group (differential validity procedures) assess the extent to which the primary factor scores relate differentially to the external variables. Procedures in the second group (incremental validity procedures) assess the extent to which the primary factor scores yield predictive validity increments with respect to the single general factor scores. Both groups of procedures are based on a second-order structural model with latent variables from which new methodological results are obtained. The functioning of the proposal is assessed by means of a simulation study, and its usefulness is illustrated with a real-data example in the personality domain.
References
|
Asparouhov, T., Muthen, B. (2010). Simple second order chi-square correction. Unpublished manuscript. Retrieved from https://www.statmodel.com/download/WLSMV_new_chi21.pdf Google Scholar | |
|
Bartlett, M. S. (1937). The statistical conception of mental factors. British Journal of Psychology, 28, 97-104. Google Scholar | |
|
Beauducel, A., Harms, C., Hilger, N. (2016). Reliability estimates for three factor score estimators. International Journal of Statistics and Probability, 5, 94-107. Google Scholar | Crossref | |
|
Betts, J., Pickart, M., Heistad, D. (2011). Investigating early literacy and numeracy: Exploring the utility of the bifactor model. School Psychology Quarterly, 26, 97-107. Google Scholar | Crossref | |
|
Carmines, E. G., Zeller, R. A. (1991). Reliability and validity assessment (Vol. 17). Newbury Park, CA: Sage. Google Scholar | |
|
Coyle, T. R., Pillow, D. R. (2008). SAT and ACT predict college GPA after removing g. Intelligence, 36, 719-729. Google Scholar | Crossref | ISI | |
|
Croon, M. (2002). Using predicted latent scores in general latent structure models. In Marcoulides, G. A., Moustaki, I. (Eds.), Latent variable and latent structure models (pp. 195-223). Mahwah, NJ: Lawrence Erlbaum. Google Scholar | |
|
Curran, P. J., Cole, V. T., Bauer, D. J., Rothenberg, W. A., Hussong, A. M. (2018). Recovering predictor–criterion relations using covariate-informed factor score estimates. Structural Equation Modeling: A Multidisciplinary Journal, 25, 860-875. doi:10.1080/1070 5511.2018.1473773 Google Scholar | Crossref | |
|
Devlieger, I., Rosseel, Y. (2017). Factor score path analysis. Methodology, 13, 31-38. Google Scholar | Crossref | |
|
Ferrando, P. J. (2008). Maximizing the information and validity of a linear composite in the factor analysis model for continuous item responses. Psicológica, 29, 189-203. Google Scholar | |
|
Ferrando, P. J., Lorenzo-Seva, U. (2000). Unrestricted versus restricted factor analysis of multidimensional test items: Some aspects of the problem and some suggestions. Psicológica, 21, 301-323. Google Scholar | |
|
Ferrando, P. J., Lorenzo-Seva, U. (2013). Unrestricted item factor analysis and some relations with item response theory (Technical Report). Tarragona, Spain: Universitat Rovira i Virgili, Department of Psychology. Google Scholar | |
|
Ferrando, P. J., Lorenzo-Seva, U. (2018a). Assessing the quality and appropriateness of factor solutions and factor score estimates in exploratory item factor analysis. Educational and Psychological Measurement, 78, 762-780. Google Scholar | SAGE Journals | ISI | |
|
Ferrando, P. J., Lorenzo-Seva, U. (2018b, May 15). On the added value of multiple factor score estimates in essentially unidimensional models. Educational and Psychological Measurement, 79, 249-271. doi:10.1177/0013164418773851 Google Scholar | SAGE Journals | |
|
Ferrando, P. J., Navarro-González, D. (2018). Assessing the quality and usefulness of factor-analytic applications to personality measures: A study with the statistical anxiety scale. Personality and Individual Differences, 123, 81-86. Google Scholar | Crossref | |
|
Floyd, F. J., Haynes, S. N., Doll, E. R., Winemiller, D., Lemsky, C., Burgy, T. M., . . . Heilman, N. (1992). Assessing retirement satisfaction and perceptions of retirement experiences. Psychology and Aging, 7, 609-621. Google Scholar | Crossref | Medline | ISI | |
|
Floyd, F. J., Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286-299. Google Scholar | Crossref | ISI | |
|
Furnham, A. (1990). The development of single trait personality theories. Personality and Individual Differences, 11, 923-929. Google Scholar | Crossref | |
|
Ghiselli, E. E., Campbell, J. P., Zedeck, S. (1981). Measurement theory for the behavioral sciences: A series of books in psychology. San Francisco. CA: W. H. Freeman. Google Scholar | |
|
Goldberg, L. R. (1972). Parameters of personality inventory construction and utilization: A comparison of prediction strategies and tactics. Multivariate Behavioral Research Monographs, 72(2), 59. Google Scholar | |
|
Gustafsson, J. E., Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28, 407-434. Google Scholar | Crossref | Medline | ISI | |
|
Hancock, G. R., Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In Cudek, R., duToit, S. H. C., Sorbom, D. F. (Eds.), Structural equation modeling: Present and future (pp. 195-216). Lincolnwood, IL: Scientific Software. Google Scholar | |
|
Haynes, S. N., Lench, H. C. (2003). Incremental validity of new clinical assessment measures. Psychological Assessment, 15, 456-466. Google Scholar | Crossref | Medline | ISI | |
|
Hoshino, T., Bentler, P. M. (2013). Bias in factor score regression and a simple solution. In de Leon, A. R., Chough, K. C. (Eds.), Analysis of mixed data (pp. 43-61). Boca Raton, FL: Chapman & Hall/CRC Press. Google Scholar | Crossref | |
|
Johnston, J. (1972). Econometric methods. New York, NY: McGraw-Hill. Google Scholar | |
|
Judge, T. A., Erez, A., Bono, J. E., Thoresen, C. J. (2002). Are measures of self-esteem, neuroticism, locus of control, and generalized self-efficacy indicators of a common core construct? Journal of Personality and Social Psychology, 83, 693-710. Google Scholar | Crossref | Medline | ISI | |
|
Lawley, D. N., Maxwell, A. E. (1963). Factor analysis as statistical method. London, England: Butterworth. Google Scholar | |
|
Lorenzo-Seva, U. (1999). Promin: A method for oblique factor rotation. Multivariate Behavioral Research, 34, 347-356. Google Scholar | Crossref | ISI | |
|
Lorenzo-Seva, U., Ferrando, P. J. (2013). FACTOR 9.2: A comprehensive program for fitting exploratory and semiconfirmatory factor analysis and IRT models. Applied Psychological Measurement, 37, 497-498. Google Scholar | SAGE Journals | ISI | |
|
McDonald, R. P. (1982). Linear versus models in item response theory. Applied Psychological Measurement, 6, 379-396. Google Scholar | SAGE Journals | ISI | |
|
McDonald, R. P. (2011). Measuring latent quantities. Psychometrika, 76, 511-536. Google Scholar | Crossref | Medline | |
|
McDonald, R. P., Burr, E. J. (1967). A comparison of four methods of constructing factor scores. Psychometrika, 32, 381-401. Google Scholar | Crossref | ISI | |
|
McNemar, Q. (1969). Psychological statistics. New York, NY: Wiley. Google Scholar | |
|
Mulaik, S. A., Quartetti, D. A. (1997). First order or higher order general factor? Structural Equation Modeling: A Multidisciplinary Journal, 4, 193-211. Google Scholar | Crossref | ISI | |
|
Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115-132. Google Scholar | Crossref | ISI | |
|
Mershon, B., Gorsuch, R. L. (1988). Number of factors in the personality sphere: Does increase in factors increase predictability of real-life criteria? Journal of Personality and Social Psychology, 55, 675-680. Google Scholar | Crossref | ISI | |
|
Morris, J. D. (1979). A comparison of regression prediction accuracy on several types of factor scores. American Educational Research Journal, 16, 17-24. Google Scholar | SAGE Journals | ISI | |
|
Nagy, G., Brunner, M., Lüdtke, O., Greiff, S. (2017). Extension procedures for confirmatory factor analysis. Journal of Experimental Education, 85, 574-596. Google Scholar | Crossref | |
|
Onwuegbuzie, A. J., Daley, C. E. (1999). Perfectionism and statistics anxiety. Personality and Individual Differences, 26, 1089-1102. Google Scholar | Crossref | ISI | |
|
Penev, S., Raykov, T. (2006). On the relationship between maximal reliability and maximal validity of linear composites. Multivariate Behavioral Research, 41, 105-126. Google Scholar | Crossref | Medline | ISI | |
|
Raykov, T., Gabler, S., Dimitrov, D. M. (2016). Maximal criterion validity and scale criterion validity: A latent variable modeling approach for examining their difference. Structural Equation Modeling: A Multidisciplinary Journal, 23, 544-554. Google Scholar | Crossref | |
|
Raykov, T., Marcoulides, G. A. (2018). On studying common factor dominance and approximate unidimensionality in multicomponent measuring instruments with discrete items. Educational and Psychological Measurement, 78, 504-516. Google Scholar | SAGE Journals | ISI | |
|
Reise, S. P., Bonifay, W. E., Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129-140. Google Scholar | Crossref | Medline | ISI | |
|
Reise, S. P., Cook, K. F., Moore, T. M. (2015). Evaluating the impact of multidimensionality on unidimensional item response theory model parameters. In Reise, S. P., Revicki, D. A. (Eds.), Handbook of item response theory modeling (pp. 13-40). New York, NY: Routledge. Google Scholar | |
|
Rindskopf, D., Rose, T. (1988). Some theory and applications of confirmatory second-order factor analysis. Multivariate Behavioral Research, 23, 51-67. Google Scholar | Crossref | Medline | ISI | |
|
Rodriguez, A., Reise, S. P., Haviland, M. G. (2016a). Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98, 223-237. Google Scholar | Crossref | Medline | ISI | |
|
Rodriguez, A., Reise, S. P., Haviland, M. G. (2016b). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21, 137. Google Scholar | Crossref | Medline | ISI | |
|
Samejima, F. (1977). Weakly parallel tests in latent trait theory with some criticism of classical test theory. Psychometrika, 42, 193-198. Google Scholar | Crossref | ISI | |
|
Steiger, J. H. (1979). The relationship between external variables and common factors. Psychometrika, 44(1), 93-97. Google Scholar | Crossref | |
|
Vigil-Colet, A., Lorenzo-Seva, U., Condon, L. (2008). Development and validation of the statistical anxiety scale. Psicothema, 20, 174-180. Google Scholar | Medline | ISI | |
|
Yuan, K. H., Chan, W., Marcoulides, G. A., Bentler, P. M. (2016). Assessing structural equation models by equivalence testing with adjusted fit indexes. Structural Equation Modeling: A Multidisciplinary Journal, 23, 319-330. Google Scholar | Crossref |
