![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Range restriction is a common problem in organizational research and is an important statistical artifact to correct for in meta-analysis. Historically, researchers have had to rely on range-restriction corrections that only make use of range-restriction information for one variable, but it is not uncommon for researchers to have such information for both variables in a correlation (e.g., when studying the correlation between two predictor variables). Existing meta-analytic methods incorporating bivariate range-restriction corrections overlook their unique implications for estimating the sampling variance of corrected correlations and for accurately assigning weights to studies in individual-correction meta-analyses. We introduce new methods for computing individual-correction and artifact-distribution meta-analyses using the bivariate indirect range restriction (BVIRR; “Case V”) correction and describe improved methods for applying BVIRR corrections that substantially reduce bias in parameter estimation. We illustrate the effectiveness of these methods in a large-scale simulation and in meta-analyses of expatriate data. We provide R code to implement the methods described in this article; more comprehensive and robust functions for applying these methods are available in the psychmeta package for R.
References
|
Aitken, A. C. (1934). Note on selection from a multivariate normal population. Proceedings of the Edinburgh Mathematical Society (Series 2), 4(2), 106–110. https://doi.org/10/bmqrbz Google Scholar | |
|
Albrecht, A.-G., Ones, D. S., Sinangil, H. K. (2018). Expatriate management. In Ones, D. S., Anderson, N., Viswesvaran, C., Sinangil, H. K. (Eds.), The SAGE handbook of industrial, work and organizational psychology (2nd ed., Vol. 3, pp. 429–465). London, UK: Sage. https://doi.org/10/c4xf Google Scholar | Crossref | |
|
Alexander, R. A. (1990). Correction formulas for correlations restricted by selection on an unmeasured variable. Journal of Educational Measurement, 27(2), 187–189. https://doi.org/10/cjrr9c Google Scholar | Crossref | ISI | |
|
Alexander, R. A., Carson, K. P., Alliger, G. M., Carr, L. (1987). Correcting doubly truncated correlations: An improved approximation for correcting the bivariate normal correlation when truncation has occurred on both variables. Educational and Psychological Measurement, 47(2), 309–315. https://doi.org/10/bc6dzj Google Scholar | SAGE Journals | ISI | |
|
Beatty, A. S., Barratt, C. L., Berry, C. M., Sackett, P. R. (2014). Testing the generalizability of indirect range restriction corrections. Journal of Applied Psychology, 99(4), 587–598. https://doi.org/10.1037/a0036361 Google Scholar | Crossref | Medline | ISI | |
|
Black, J. S., Gregersen, H. B. (1991). The other half of the picture: Antecedents of spouse cross-cultural adjustment. Journal of International Business Studies, 22(3), 461–477. https://doi.org/10/b2nbxx Google Scholar | Crossref | |
|
Bryant, N. D., Gokhale, S. (1972). Correcting correlations for restrictions in range due to selection on an unmeasured variable. Educational and Psychological Measurement, 32(2), 305–310. https://doi.org/10/fc4533 Google Scholar | SAGE Journals | ISI | |
|
Colquitt, J. A., Scott, B. A., Rodell, J. B., Long, D. M., Zapata, C. P., Conlon, D. E., Wesson, M. J. (2013). Justice at the millennium, a decade later: A meta-analytic test of social exchange and affect-based perspectives. Journal of Applied Psychology, 98(2), 199–236. https://doi.org/10/q6c Google Scholar | Crossref | Medline | ISI | |
|
Dahlke, J. A., Wiernik, B. M. (2018). psychmeta: An R package for psychometric meta-analysis. Applied Psychological Measurement, 43(5), 415–416. https://doi.org/10/gfgt9t Google Scholar | SAGE Journals | |
|
Dahlke, J. A., Wiernik, B. M. (2019). psychmeta: Psychometric meta-analysis toolkit [R Package]. Retrieved from https://cran.r-project.org/package=psychmeta (Original work published 2017) Google Scholar | |
|
DeRue, D. S., Nahrgang, J. D., Wellman, N. E. D., Humphrey, S. E. (2011). Trait and behavioral theories of leadership: An integration and meta-analytic test of their relative validity. Personnel Psychology, 64(1), 7–52. https://doi.org/10/fwzt2t Google Scholar | Crossref | ISI | |
|
Elwert, F., Winship, C. (2014). Endogenous selection bias: The problem of conditioning on a collider variable. Annual Review of Sociology, 40(1), 31–53. https://doi.org/10/gdnb59 Google Scholar | Crossref | Medline | |
|
Harari, M. B., Reaves, A. C., Beane, D. A., Laginess, A. J., Viswesvaran, C. (2018). Personality and expatriate adjustment: A meta-analysis. Journal of Occupational and Organizational Psychology, 91(3), 486–517. https://doi.org/10/gdnb6c Google Scholar | Crossref | |
|
Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161. https://doi.org/10/c62z76 Google Scholar | Crossref | ISI | |
|
Humphrey, S. E., Nahrgang, J. D., Morgeson, F. P. (2007). Integrating motivational, social, and contextual work design features: A meta-analytic summary and theoretical extension of the work design literature. Journal of Applied Psychology, 92(5), 1332–1356. https://doi.org/10/dch52t Google Scholar | Crossref | Medline | ISI | |
|
Hunter, J. E., Schmidt, F. L., Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91(3), 594–612. https://doi.org/10/bt4t68 Google Scholar | Crossref | Medline | ISI | |
|
Jones, J. A., Waller, N. G. (2013). Computing confidence intervals for standardized regression coefficients. Psychological Methods, 18(4), 435–453. https://doi.org/10/gckfx4 Google Scholar | Crossref | Medline | |
|
Kostal, J. W., Wiernik, B. M., Albrecht, A.-G., Ones, D. S. (2018). Expatriate personality: Facet-level comparisons with domestic counterparts. In Wiernik, B. M., Rüger, H., Ones, D. S. (Eds.), Managing expatriates: Success factors in private and public domains (pp. 33–52). Opladen, Germany: Budrich. https://doi.org/10/c4xg Google Scholar | |
|
Lawley, D. N. (1943). A note on Karl Pearson’s selection formulae. Proceedings of the Royal Society of Edinburgh. Section A. Mathematical and Physical Sciences, 62(1), 28–30. https://doi.org/10/ckc2 Google Scholar | |
|
Le, H., Oh, I.-S., Schmidt, F. L., Wooldridge, C. D. (2016). Correction for range restriction in meta-analysis revisited: Improvements and implications for organizational research. Personnel Psychology, 69(4), 975–1008. https://doi.org/10/gckf2p Google Scholar | Crossref | |
|
Le, H., Schmidt, F. L. (2006). Correcting for indirect range restriction in meta-analysis: Testing a new meta-analytic procedure. Psychological Methods, 11(4), 416–438. https://doi.org/10/fv7g5s Google Scholar | Crossref | Medline | ISI | |
|
Meyer, S. L. (1975). Data analysis for scientists and engineers. New York, NY: John Wiley. Google Scholar | |
|
Murray, A. L., Johnson, W., McGue, M., Iacono, W. G. (2014). How are conscientiousness and cognitive ability related to one another? A re-examination of the intelligence compensation hypothesis. Personality and Individual Differences, 70, 17–22. https://doi.org/10/f6gvt6 Google Scholar | Crossref | |
|
Oehlert, G. W. (1992). A note on the delta method. American Statistician, 46(1), 27–29. https://doi.org/10/d27qpx Google Scholar | ISI | |
|
Ones, D. S., Sinangil, H. K., Wiernik, B. M. (2018). Validity of Big Five personality traits for expatriate success: Results from Turkey. In Wiernik, B. M., Rüger, H., Ones, D. S. (Eds.), Managing expatriates: Success factors in private and public domains (pp. 83–102). Opladen, Germany: Budrich. https://doi.org/10/c4xh Google Scholar | |
|
Ones, D. S., Viswesvaran, C. (2003). Job-specific applicant pools and national norms for personality scales: Implications for range-restriction corrections in validation research. Journal of Applied Psychology, 88(3), 570–577. https://doi.org/10/c2rvxw Google Scholar | Crossref | Medline | |
|
Oswald, F. L., McCloy, R. A. (2003). Meta-analysis and the art of the average. In Murphy, K. R. (Ed.), Validity generalization: A critical review (pp. 311–338). Mahwah, NJ: Lawrence Erlbaum. https://doi.org/10/ckc3 Google Scholar | |
|
Raju, N. S., Burke, M. J. (1983). Two new procedures for studying validity generalization. Journal of Applied Psychology, 68(3), 382–395. https://doi.org/10/fq5thj Google Scholar | Crossref | |
|
Raju, N. S., Burke, M. J., Normand, J., Langlois, G. M. (1991). A new meta-analytic approach. Journal of Applied Psychology, 76(3), 432–446. https://doi.org/10/dcrgkf Google Scholar | Crossref | |
|
Rohrer, J. M. (2018). Thinking clearly about correlations and causation: Graphical causal models for observational data. Advances in Methods and Practices in Psychological Science, 1(1), 27–42. https://doi.org/10/gcvj3r Google Scholar | SAGE Journals | |
|
Rousseau, D. M., Fried, Y. (2001). Location, location, location: Contextualizing organizational research. Journal of Organizational Behavior, 22(1), 1–13. https://doi.org/10/dcvv7s Google Scholar | Crossref | ISI | |
|
Sackett, P. R., Lievens, F., Berry, C. M., Landers, R. N. (2007). A cautionary note on the effects of range restriction on predictor intercorrelations. Journal of Applied Psychology, 92(2), 538–544. https://doi.org/10/ch6qk2 Google Scholar | Crossref | Medline | |
|
Sackett, P. R., Ostgaard, D. J. (1994). Job-specific applicant pools and national norms for cognitive ability tests: Implications for range restriction corrections in validation research. Journal of Applied Psychology, 79(5), 680–684. https://doi.org/10/fkpnv2 Google Scholar | Crossref | Medline | |
|
Sackett, P. R., Yang, H. (2000). Correction for range restriction: An expanded typology. Journal of Applied Psychology, 85(1), 112–118. https://doi.org/10/c6npmd Google Scholar | Crossref | Medline | ISI | |
|
Schmidt, F. L., Hunter, J. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Thousand Oaks, CA: Sage. https://doi.org/10/b6mg Google Scholar | Crossref | |
|
Stein, S. K., Barcellos, A. (1992). Calculus and analytic geometry (5th ed.). New York, NY: McGraw-Hill. Google Scholar | |
|
Wiernik, B. M., Dahlke, J. A. (in press). Obtaining unbiased results in meta-analysis: The importance of correcting for statistical artefacts. Advances in Methods and Practices in Psychological Science. Google Scholar | |
|
Wiernik, B. M., Kostal, J. W. (2019). Protean and boundaryless career orientations: A critical review and meta-analysis. Journal of Counseling Psychology, 66(3), 280–307. https://doi.org/10/gfxrgp Google Scholar | |
|
Wiernik, B. M., Kostal, J. W., Wilmot, M. P., Dilchert, S., Ones, D. S. (2017). Empirical benchmarks for interpreting effect size variability in meta-analysis. Industrial and Organizational Psychology, 10(3), 472–479. https://doi.org/10/ccnv Google Scholar | Crossref |
