![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Researchers are becoming interested in combining meta-analytic techniques and structural equation modeling to test theoretical models from a pool of studies. Most existing procedures are based on the assumption that all correlation matrices are homogeneous. Few studies have addressed what the next step should be when studies being analyzed are heterogeneous and the search for moderator variables for homogeneous subgroup analysis fails. Cluster analysis is proposed and evaluated in this article as an exploratory tool to classify studies into relatively homogeneous groups. Simulation studies indicate that using Euclidean distance on raw correlation coefficients or U-transformed scores with the complete linkage or Ward’s minimum-variance methods will provide satisfactory results.
|
Aldenderfer, M. S. , & Blashfield, R. K. (1984). Cluster analysis. Beverly Hills, CA: Sage. Google Scholar | Crossref | |
|
Bacon, D. R. (2001). An evaluation of cluster analytic approaches to initial model specification. Structural Equation Modeling, 8, 397-429. Google Scholar | Crossref | |
|
Becker, B. J. (1992). Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 17, 341-362. Google Scholar | Crossref | |
|
Becker, B. J. (1995). Corrections to “Using results from replicated studies to estimate linear models.” Journal of Educational Statistics, 20, 100-102. Google Scholar | SAGE Journals | |
|
Becker, B. J. (2000). Multivariate meta-analysis. In H. E. A. Tinsley & S. D. Brown (Eds.) Handbook of applied multivariate statistics and mathematical modeling (pp.499-525). San Diego, CA: Academic Press. Google Scholar | Crossref | |
|
Becker, B. J. (2001). Examining theoretical models through research synthesis: The benefits of model-driven meta-analysis. Evaluation and the Health Professions, 24, 190-217. Google Scholar | SAGE Journals | |
|
Becker, B. J. , & Schram, C. M. (1994). Examining explanatory models through research synthesis. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 357-381). New York: Russell Sage. Google Scholar | |
|
Blashfield, R. K. , & Aldenderfer, M. S. (1989). The methods and problems of cluster analysis. In J. R. Nesselroade & R. B. Cattell (Eds.), Handbook of multivariate experimental psychology (pp. 447-473). New York: Plenum. Google Scholar | |
|
Brown, S. P. , & Peterson, R. A. (1993). Antecedents and consequences of salesperson job satisfaction: meta-analysis and assessment of causal effects. Journal of Marketing Research, 30, 63-77. Google Scholar | Crossref | |
|
Brown, S. P. , & Stayman, D. M. (1992). Antecedents and consequences of attitude toward the ad: A meta-analysis. Journal of Consumer Research, 19, 34-51. Google Scholar | Crossref | ISI | |
|
Cheung, M. W. L. , & Chan, W. (2005). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10, 40-64. Google Scholar | Crossref | Medline | ISI | |
| Cheung, S. F. (2000). Examining solutions to two practical issues in meta-analysis: Dependent correlations and missing data in correlation matrices (Doctoral dissertation, Chinese University of Hong Kong, 2000). Dissertation Abstracts International, 61(8-B), 4469. Google Scholar | |
|
Cohen, J. (1960). A coefficient of agreement for normal scales. Educational and Psychological Measurement, 20, 37-46. Google Scholar | SAGE Journals | ISI | |
|
Colquitt, J. A. , LePine, J. A. , & Noe, R. A. (2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, 678-707. Google Scholar | Crossref | Medline | ISI | |
|
Cortina, J. M. (2003). Apples and oranges (and pears, oh my!): The search for moderators in meta-analysis. Organizational Research Methods, 6, 415-439. Google Scholar | SAGE Journals | ISI | |
|
Digman, J. M. (1997). Higher-order factors of the Big Five. Journal of Personality and Social Psychology, 73, 1246-1256. Google Scholar | Crossref | Medline | ISI | |
|
Dumenci, L. , & Windle, M. (2001). Cluster analysis as a method of recovering types of intraindividual growth trajectories: A Monte Carlo study. Multivariate Behavioral Research, 36, 501-522. Google Scholar | Crossref | Medline | |
|
Engels, E. A. , Schmid, C. H. , Terrin, N. , Olkin, I. , & Lau, S. (2000). Heterogeneity and statistical significance in meta-analysis: An empirical study of 125 meta-analyses. Statistics in Medicine, 19, 1707-1728. Google Scholar | Crossref | Medline | |
|
Everitt, B. S. , Landau, S. , & Leese, M. (2001). Cluster analysis (4th ed.). London: Oxford University Press. Google Scholar | |
|
Field, A. P. (2001). Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed- and random-effects methods. Psychological Methods, 6, 161-180. Google Scholar | Crossref | Medline | ISI | |
|
Glass, G. V (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3-8. Google Scholar | SAGE Journals | |
|
Glasziou, P. P. , & Sanders, S. L. (2002). Investigating causes of heterogeneity in systematic reviews. Statistics in Medicine, 21, 1503-1511. Google Scholar | Crossref | Medline | |
|
Gordon, A. D. (1998).Cluster validation. In C. Hayashi , N. Ohsumi , K. Yajima , Y. Tanaka , H.-H. Bock , & Y. Baba (Eds.), Data science: Classification and related methods (pp. 22-39). New York: Springer-Verlag. Google Scholar | Crossref | |
|
Gordon, A. D. (1999). Classification (2nd ed.). London: Chapman & Hall. Google Scholar | |
| Hafdahl, A. R. (2002). Multivariate meta-analysis for exploratory factor analytic research (Doctoral dissertation, University of North Carolina at Chapel Hill, 2002). Dissertation Abstracts International, 62(8-B), 3843. Google Scholar | |
|
Hedges, L. V. , & Olkin, I. (1983). Clustering estimates of effect magnitude from independent studies. Psychological Bulletin, 93(3), 563-573. Google Scholar | Crossref | |
|
Hedges, L. V. , & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press. Google Scholar | |
|
Hedges, L. V. , & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486-504. Google Scholar | Crossref | ISI | |
|
Hubert, L. , & Arabie, P. (1985). Comparing partitions. Journal of Classification, 2, 193-218. Google Scholar | Crossref | ISI | |
|
Hunter, J. E. , & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage. Google Scholar | |
|
Jardine, N. , & Sibson, R. (1971). Mathematical taxonomy. New York: John Wiley. Google Scholar | |
|
Kalaian, H. A. , & Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis. Psychological Methods, 1, 227-235. Google Scholar | Crossref | ISI | |
|
Lance, G. N. , & Williams, W. T. (1967). A general theory of classificatory sorting strategies. Computer Journal, 9, 373-380. Google Scholar | Crossref | |
|
Lijmer, J. G. , Bossuyt, P. M. M. , & Heisterkamp, S. H. (2002). Exploring sources of heterogeneity in systematic reviews of diagnostic tests. Statistics in Medicine, 21, 1525-1537. Google Scholar | Crossref | Medline | |
|
Mardia, K. V. , Kent, J. T. , & Bibby, J. M. (1979). Multivariate analysis. London: Academic Press. Google Scholar | |
|
Marín-Martínez, F. , & Sánchez-Meca, J. (1998). Testing for dichotomous moderators in meta-analysis. Journal of Experimental Education, 67, 69-81. Google Scholar | Crossref | |
|
Miller, N. , & Pollock, V. E. (1994). Meta-analytic synthesis for theory development. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 457-483). New York: Russell Sage. Google Scholar | |
|
Milligan, G. W. , & Cooper, M. C. (1985). An examination of procedures for determining the number of cluster in a data set. Psychometrika, 50, 159-179. Google Scholar | Crossref | |
|
Milligan, G. W. , & Cooper, M. C. (1986). A study of the comparability of external criteria for hierarchical cluster analysis. Multivariate Behavioral Research, 21, 41-58. Google Scholar | Crossref | Medline | |
|
Milligan, G. W. , & Cooper, M. C. (1987). Methodology review: Clustering methods. Applied Psychological Measurement, 11, 329-354. Google Scholar | SAGE Journals | ISI | |
|
Milligan, G. W. , & Cooper, M. C. (1988). A study of standardization of variables in cluster analysis. Journal of Classification, 5, 181-204. Google Scholar | Crossref | ISI | |
|
Milligan, G. W. , & Isaac, P. (1980). The validation of four ultrametric clustering algorithms. Pattern Recognition, 12, 41-50. Google Scholar | Crossref | ISI | |
|
Muthén, B. (2004). Latent variable analysis: Growth mixture modeling and related techniques for longitudinal data. In D. Kaplan (Ed.), Handbook of quantitative methodology for the social sciences (pp. 345-368). Newbury Park, CA: Sage. Google Scholar | Crossref | |
|
Muthén, B. , Brown, C. H. , Masyn, K. , Jo, B. , Khoo, S. T. , Yang, C. C. , et al. (2002). General growth mixture modeling for randomized preventive interventions. Biostatistics, 3, 459-475. Google Scholar | Crossref | Medline | ISI | |
|
National Research Council . (1992). Combining information: Statistical issues and opportunities for research. Washington, DC: National Academy Press. Google Scholar | |
|
Overall, J. E. , & Magee, K. N. (1992). Replication as a rule for determining the number of clusters in hierarchical cluster analysis. Applied Psychological Measurement, 16, 119-128. Google Scholar | SAGE Journals | ISI | |
|
Pigott, T. D. (2001). Missing predictors in models of effect size. Evaluation and the Health Professionals, 24, 277-307. Google Scholar | SAGE Journals | |
|
Poon, W. Y. , Chan, W. , Lee, S. Y. , & Leung, K. (1993). Preliminary analysis of multiple group structural equation modeling via cluster analysis. In American Statistical Association (Ed.), Proceedings of the Social Statistics Section (pp. 368-373). Washington, DC: American Statistical Association. Google Scholar | |
|
Raudenbush, S. W. , Becker, B. J. , & Kalaian, H. (1988). Modeling multivariate effect sizes. Psychological Bulletin, 103, 111-120. Google Scholar | Crossref | ISI | |
|
Roussos, L. A. , Stout, W. F. , & Marden, J. I. (1998). Using new proximity measures with hierarchical cluster analysis to detect multidimensionality. Journal of Educational Measurement, 35, 1-30. Google Scholar | Crossref | ISI | |
|
SAS Institute . (1995). SAS/IML software: Changes and enhancements, through Release 6.11. Cary, NC: Author. Google Scholar | |
|
Shadish, W. R. (1996). Meta-analysis and the exploration of causal mediating processes: A primer of examples, methods, and issues. Psychological Methods, 1, 47-65. Google Scholar | Crossref | ISI | |
|
Sokal, R. R. , & Michener, C. D. (1958). A statistical method for evaluating systematic relationships. University of Kansas Science Bulletin, 38, 1409-1438. Google Scholar | |
|
Sokal, R. R. , & Sneath, P. H. (1963). Principles of numerical taxonomy. London: Freeman. Google Scholar | |
|
Song, F. , Sheldon, T. A. , Sutton, A. J. , & Jones, D. R. (2001). Methods for exploring heterogeneity in meta-analysis. Evaluation and the Health Professions, 24, 126-151. Google Scholar | SAGE Journals | ISI | |
|
Verhaeghen, P. , & Salthouse, T. A. (1997). Meta-analyses of age-cognition relations in adulthood: Estimates of linear and nonlinear age effects and structural models. Psychological Bulletin, 122, 231-249. Google Scholar | Crossref | Medline | ISI | |
|
Viswesvaran, C. , & Ones, D. S. (1995). Theory testing: Combining psychometric meta-analysis and structural equations modeling. Personnel Psychology, 48, 865-885. Google Scholar | Crossref | ISI | |
|
Viswesvaran, C. , & Sanchez, J. I. (1998). Moderator search in meta-analysis: A review and cautionary note on existing approaches. Educational and Psychological Measurement, 58, 77-87. Google Scholar | SAGE Journals | ISI |
