![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
The current conventions for test score reliability coefficients are unsystematic and chaotic. Reliability coefficients have long been denoted using names that are unrelated to each other, with each formula being generated through different methods, and they have been represented inconsistently. Such inconsistency prevents organizational researchers from understanding the whole picture and misleads them into using coefficient alpha unconditionally. This study provides a systematic naming convention, formula-generating methods, and methods of representing each of the reliability coefficients. This study offers an easy-to-use solution to the issue of choosing between coefficient alpha and composite reliability. This study introduces a calculator that enables its users to obtain the values of various multidimensional reliability coefficients with a few mouse clicks. This study also presents illustrative numerical examples to provide a better understanding of the characteristics and computations of reliability coefficients.
References
|
Angoff, W. H. (1953). Test reliability and effective test length. Psychometrika, 18(1), 1–14. Google Scholar | Crossref | |
|
Armor, D. J. (1974). Theta reliability and factor scaling. In Costner, H. L. (Ed.), Sociological methodology (pp. 17–50). San Francisco, CA: Jossey-Bass. Google Scholar | |
|
Bentler, P. M. (2006). EQS 6 structural equations program manual. Encino, CA: Multivariate Software. Google Scholar | |
|
Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74(1), 137–143. Google Scholar | Crossref | Medline | ISI | |
|
Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: John Wiley. Google Scholar | Crossref | |
|
Brown, W. (1910). Some experimental results in the correlation of mental abilities. British Journal of Psychology, 3(3), 296–322. Google Scholar | |
|
Brunner, M., Nagy, G., Wilhelm, O. (2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80(4), 796–846. Google Scholar | Crossref | Medline | ISI | |
|
Cho, E., Kim, S. (2015). Cronbach’s coefficient alpha: Well known but poorly understood. Organizational Research Methods, 18(2), 207–230. Google Scholar | SAGE Journals | ISI | |
|
Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98–104. Google Scholar | Crossref | ISI | |
|
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. Google Scholar | Crossref | |
|
Cronbach, L. J. (1978). Citation classics. Current Contents, 13, 263. Google Scholar | |
|
Cronbach, L. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64(3), 391–418. Google Scholar | SAGE Journals | ISI | |
|
Dunn, T. J., Baguley, T., Brunsden, V. (2013). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. Google Scholar | Crossref | Medline | ISI | |
|
Falk, C. F., Savalei, V. (2011). The relationship between unstandardized and standardized alpha, true reliability, and the underlying measurement model. Journal of Personality Assessment, 93(5), 445–453. Google Scholar | Crossref | Medline | ISI | |
|
Feldt, L. S. (1975). Estimation of the reliability of a test divided into two parts of unequal length. Psychometrika, 40(4), 557–561. Google Scholar | Crossref | ISI | |
|
Feldt, L. S., Brennan, R. L. (1989). Reliability. In Linn, R. L. (Ed.), Educational measurement (3rd ed., pp. 105–146). New York, NY: American Council on Education and Macmillan. Google Scholar | |
|
Fisher, R. A. (1925). Statistical methods for research workers. Edinburgh: Oliver and Boyd Ltd. Google Scholar | |
|
Flanagan, J. C. (1937). A proposed procedure for increasing the efficiency of objective tests. Journal of Educational Psychology, 28(1), 17–21. Google Scholar | Crossref | |
|
Graham, J. M. (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and how to use them. Educational and Psychological Measurement, 66(6), 930–944. Google Scholar | SAGE Journals | ISI | |
|
Green, S. B., Lissitz, R. W., Mulaik, S. A. (1977). Limitations of coefficient alpha as an index of test unidimensionality. Educational and Psychological Measurement, 37(4), 827–838. Google Scholar | SAGE Journals | ISI | |
|
Green, S. B., Yang, Y. (2009a). Commentary on coefficient alpha: A cautionary tale. Psychometrika, 74(1), 121–135. Google Scholar | Crossref | ISI | |
|
Green, S. B., Yang, Y. (2009b). Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74(1), 155–167. Google Scholar | Crossref | ISI | |
|
Green, S. B., Yang, Y. (2015). Evaluation of dimensionality in the assessment of internal consistency reliability: Coefficient alpha and omega coefficients. Educational Measurement: Issues and Practice, 34(4), 14–20. Google Scholar | Crossref | ISI | |
|
Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4), 255–282. Google Scholar | Crossref | Medline | |
|
Haertel, E. H. (2006). Reliability. In Brennan, R. L. (Ed.), Educational measurement (4th ed., pp. 65–110). Westport, CT: American Council on Education and Praeger. Google Scholar | |
|
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E. (2010). Multivariate data analysis. Upper Saddle River, NJ: Pearson. Google Scholar | |
|
Hattie, J. (1985). Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9(2), 139–164. Google Scholar | SAGE Journals | ISI | |
|
Hoyt, C. (1941). Test reliability estimated by analysis of variance. Psychometrika, 6(3), 153–160. Google Scholar | Crossref | |
|
Hunt, T. D., Bentler, P. M. (2015). Quantile lower bounds to reliability based on locally optimal splits. Psychometrika, 80(1), 182–195. Google Scholar | Crossref | Medline | ISI | |
|
Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36(2), 109–133. Google Scholar | Crossref | ISI | |
|
Kamata, A., Turhan, A., Darandari, E. (2003, April). Estimating reliability for multidimensional composite scale scores. Paper presented at the annual meeting of American Educational Research Association, Chicago, IL. Google Scholar | |
|
Kuder, G. F., Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2(3), 151–160. Google Scholar | Crossref | |
|
Lance, C. E., Butts, M. M., Michels, L. C. (2006). The sources of four commonly reported cutoff criteria. Organizational Research Methods, 9(2), 202–220. Google Scholar | SAGE Journals | ISI | |
|
Lord, F. M., Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley. Google Scholar | |
|
Lucke, J. F. (2005). “Rassling the hog”: The influence of correlated item error on internal consistency, classical reliability and congeneric reliability. Applied Psychological Measurement, 29(2), 106–125. Google Scholar | SAGE Journals | ISI | |
|
McDonald, R. P. (1978). Generalizability in factorable domains: “Domain validity and generalizability”: 1. Educational and Psychological Measurement, 38(1), 75–79. Google Scholar | SAGE Journals | ISI | |
|
McDonald, R. P. (1985). Factor analysis and related methods. Hillsdale, NJ: Lawrence Erlbaum. Google Scholar | |
|
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum. Google Scholar | |
|
Miles, J. N. V. (2005). Confirmatory factor analysis using Microsoft Excel. Behavior Research Methods, 37(4), 672–676. Google Scholar | Crossref | Medline | ISI | |
|
Miller, M. B. (1995). Coefficient alpha: A basic introduction from the perspectives of classical test theory and structural equation modeling. Structural Equation Modeling, 2(3), 255–273. Google Scholar | Crossref | ISI | |
|
Mosier, C. I. (1941). A short cut in the estimation of split-halves coefficients. Educational and Psychological Measurement, 1(1), 407–408. Google Scholar | SAGE Journals | |
|
Novick, M. R., Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32(1), 1–13. Google Scholar | Crossref | Medline | ISI | |
|
Osburn, H. G. (2000). Coefficient alpha and related internal consistency reliability coefficients. Psychological Methods, 5(3), 343–355. Google Scholar | Crossref | Medline | ISI | |
|
Padilla, M. A., Divers, J. (2013). Coefficient omega bootstrap confidence intervals: Nonnormal distributions. Educational and Psychological Measurement, 73(6), 956–972. Google Scholar | SAGE Journals | ISI | |
|
Peters, G. J. Y. (2014). The alpha and the omega of scale reliability and validity. European Health Psychologist, 16(2), 56–69. Google Scholar | |
|
Peterson, R. A., Kim, Y. (2013). On the relationship between coefficient alpha and composite reliability. Journal of Applied Psychology, 98(1), 194–198. Google Scholar | Crossref | Medline | ISI | |
|
Rae, G. (2008). A note on using alpha and stratified alpha to estimate the reliability of a test composed of item parcels. British Journal of Mathematical and Statistical Psychology, 61(2), 515–525. Google Scholar | Crossref | Medline | ISI | |
|
Rajaratnam, N., Cronbach, L. J., Gleser, G. C. (1965). Generalizability of stratified-parallel tests. Psychometrika, 30(1), 39–56. Google Scholar | Crossref | Medline | ISI | |
|
Raju, N. S. (1970). New formula for estimating total test reliability from parts of unequal length. Proceedings of the 78th Annual Convention of APA, 5, 143–144. Google Scholar | |
|
Raju, N. S. (1977). A generalization of coefficient alpha. Psychometrika, 42(4), 549–565. Google Scholar | Crossref | ISI | |
|
Raykov, T. (2012). Scale construction and development using structural equation modeling. In Hoyle, R. H. (Eds.), Handbook of structural equation modeling (pp. 472–492). New York, NY: Guilford. Google Scholar | |
|
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. Google Scholar | Crossref | Medline | ISI | |
|
Revelle, W. (1979). Hierarchical cluster-analysis and the internal structure of tests. Multivariate Behavioral Research, 14(1), 57–74. Google Scholar | Crossref | Medline | ISI | |
|
Revelle, W., Zinbarg, R. E. (2009). Coefficients alpha, beta, omega and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. Google Scholar | Crossref | ISI | |
|
Rulon, P. J. (1939). A simplified procedure for determining the reliability of a test by split-halves. Harvard Educational Review, 9, 99–103. Google Scholar | |
|
Schmid, J., Leiman, J. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22(1), 53–61. Google Scholar | Crossref | ISI | |
|
Schmidt, F. L., Le, H., Ilies, R. (2003). Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual differences constructs. Psychological Methods, 8(2), 206–224. Google Scholar | Crossref | Medline | ISI | |
|
Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8(4), 350–353. Google Scholar | Crossref | ISI | |
|
Sijtsma, K. (2009a). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120. Google Scholar | Crossref | Medline | ISI | |
|
Sijtsma, K. (2009b). Reliability beyond theory and into practice. Psychometrika, 74(1), 169–173. Google Scholar | Crossref | Medline | ISI | |
|
Spearman, C. (1910). Correlation calculated from faulty data. British Journal Psychology, 3(3), 271–295. Google Scholar | |
|
Tang, W., Cui, Y. (2012, April). A simulation study for comparing three lower bounds to reliability. Paper presented at the annual meeting of the American Educational Research Association, Vancouver, Canada. Google Scholar | |
|
Ten Berge, J. M. F., Zegers, F. E. (1978). A series of lower bounds to the reliability of a test. Psychometrika, 43, 575–579. Google Scholar | Crossref | ISI | |
|
van der Ark, L. A., van der Palm, D. W., Sijtsma, K. (2011). A latent class approach to estimating test-score reliability. Applied Psychological Measurement, 35(5), 380–392. Google Scholar | SAGE Journals | ISI | |
|
Werts, C. E., Rock, D. R., Linn, R. L., Jöreskog, K. G. (1978). A general method of estimating the reliability of a composite. Educational and Psychological Measurement, 38(4), 933–938. Google Scholar | SAGE Journals | ISI | |
|
Yang, Y., Green, S. B. (2011). Coefficient alpha: A reliability coefficient for the 21st century? Journal of Psychoeducational Assessment, 29(4), 377–392. Google Scholar | SAGE Journals | ISI | |
|
Yung, Y. F., Thissen, D., McLeod, L. D. (1999). On the relationship between the higher-order factor model and the hierarchical factor model. Psychometrika, 64(2), 113–128. Google Scholar | Crossref | ISI | |
|
Zinbarg, R. E., Revelle, W., Yovel, I. (2007). Estimating ωh for structures containing two group factors: Perils and prospects. Applied Psychological Measurement, 31(2), 135–157. Google Scholar | SAGE Journals | ISI | |
|
Zinbarg, R. E., Revelle, W., Yovel, I., Li, W. (2005). Cronbach’s α, Revelle’s β, and McDonald’s ωH: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1), 123–133. Google Scholar | Crossref | ISI | |
|
Zinbarg, R. E., Yovel, I., Revelle, W., McDonald, R. P. (2006). Estimating generalizability to a latent variable common to all of a scale’s indicators: A comparison of estimators for ωh. Applied Psychological Measurement, 30(2), 121–144. Google Scholar | SAGE Journals | ISI |
