![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Cross-level interaction effects lay at the heart of multilevel contingency and interactionism theories. Also, practitioners are particularly interested in such effects because they provide information on the contextual conditions and processes under which interventions focused on individuals (e.g., selection, leadership training, performance appraisal, and management) result in more or less positive outcomes. We derive a new intraclass correlation, ρβ, to assess the degree of lower-level outcome variance that is attributed to higher-level differences in slope coefficients. We provide analytical and empirical evidence that ρβ is an index of variance that differs from the traditional intraclass correlation ρα and use data from recently published articles to illustrate that ρα assesses differences across collectives and higher-level processes (e.g., teams, leadership styles, reward systems) but ignores the variance attributed to differences in lower-level relationships (e.g., individual level job satisfaction and individual level performance). Because ρα and ρβ provide information on two different sources of variability in the data structure (i.e., differences in means and differences in relationships, respectively), our results suggest that researchers contemplating the use of multilevel modeling, as well those who suspect nonindependence in their data structure, should expand the decision criteria for using multilevel approaches to include both types of intraclass correlations. To facilitate this process, we offer an illustrative data set and the icc beta R package for computing ρβ in single- and multiple-predictor situations and make them available through the Comprehensive R Archive Network (i.e., CRAN).
References
|
Aguinis, H. (2004). Regression analysis for categorical moderators. New York, NY: Guilford. Google Scholar | |
|
Aguinis, H., Beaty, J. C., Boik, R. J., Pierce, C. A. (2005). Effect size and power in assessing moderating effects of categorical variables using multiple regression: A 30-year review. Journal of Applied Psychology, 90, 94–107. Google Scholar | Crossref | Medline | ISI | |
|
Aguinis, H., Gottfredson, R. K., Culpepper, S. A. (2013). Best-practice recommendations for estimating cross-level interaction effects using multilevel modeling. Journal of Management, 39, 1490–1528. Google Scholar | SAGE Journals | ISI | |
|
Aguinis, H., Pierce, C. A., Bosco, F. A., Muslin, I. S. (2009). First decade of Organizational Research Methods: Trends in design, measurement, and data-analysis topics. Organizational Research Methods, 12, 69–112. Google Scholar | SAGE Journals | ISI | |
|
Aguinis, H., Stone-Romero, E. F. (1997). Methodological artifacts in moderated multiple regression and their effects on statistical power. Journal of Applied Psychology, 82, 192–206. Google Scholar | Crossref | ISI | |
|
Aguinis, H., Werner, S., Abbott, J. L., Angert, C., Park, J. H., Kohlhausen, D. (2010). Customer-centric science: Reporting significant research results with rigor, relevance, and practical impact in mind. Organizational Research Methods, 13, 515–539. Google Scholar | SAGE Journals | ISI | |
|
Bosco, F. A., Aguinis, H., Singh, K., Field, J. G., Pierce, C. A. (in press). Correlational effect size benchmarks. Journal of Applied Psychology. doi: http://dx.doi.org/10.1037/a0038047 Google Scholar | ISI | |
|
Burstein, L., Linn, R. L., Capell, F. J. (1978). Analyzing multilevel data in the presence of heterogeneous within-class regressions. Journal of Educational Statistics, 3, 347–383. Google Scholar | SAGE Journals | |
|
Cao, J., Ramsay, J. O. (2010). Linear mixed-effects modeling by parameter cascading. Journal of the American Statistical Association, 105, 365–374. Google Scholar | Crossref | ISI | |
|
Chen, G., Kirkman, B. L., Kanfer, R., Allen, D., Rosen, B. (2007). A multilevel study of leadership, empowerment, and performance in teams. Journal of Applied Psychology, 92, 331–346. Google Scholar | Crossref | Medline | ISI | |
|
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Google Scholar | |
|
Cortina, J. M., Landis, R. S. (2009). When small effect sizes tell a big story, and when large effect sizes don’t. In Lance, C. E., Vandenberg, R. J. (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity, and fable in the organizational and social sciences (pp. 287–308). New York, NY: Routledge. Google Scholar | |
|
Crainiceanu, C., Ruppert, D. (2004). Likelihood ratio tests in linear mixed models with one variance component. Journal of the Royal Statistical Society-B, 66, 165–185. Google Scholar | Crossref | |
|
Dalal, D. K., Zickar, M. J. (2012). Some common myths about centering predictor variables in moderated multiple regression and polynomial regression. Organizational Research Methods, 15, 339–362. Google Scholar | SAGE Journals | ISI | |
|
Enders, C. K., Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12, 121–138. Google Scholar | Crossref | Medline | ISI | |
|
Halbesleben, J. R. B., Wheeler, A. R., Paustian-Underdahl, S. C. (2013). The impact of furloughs on emotional exhaustion, self-rated performance, and recovery experiences. Journal of Applied Psychology, 98, 492–503. Google Scholar | Crossref | Medline | ISI | |
|
Heck, R. H., Thomas, S. L., Tabata, L. N. (2010). Multilevel and longitudinal modeling with IBM SPSS. New York, NY: Routledge. Google Scholar | |
|
Hofmann, D. A., Gavin, M. B. (1998). Centering decisions in hierarchical linear models: Theoretical and methodological implications for organizational science. Journal of Management, 24, 623–641. Google Scholar | SAGE Journals | ISI | |
|
Hofmann, D.A., Griffin, M. A., Gavin, M.B. (2000). The application of hierarchical linear modeling to management research. In Klein, K. J., Kozlowski, S. W. J. (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 467–511). Hoboken, NJ: Jossey-Bass. Google Scholar | |
|
Hofmann, D. A., Morgeson, F. P., Gerras, S. J. (2003). Climate as a moderator of the relationship between leader-member exchange and content specific citizenship: Safety climate as an exemplar. Journal of Applied Psychology, 88, 170–178. Google Scholar | Crossref | Medline | ISI | |
|
Hox, J. J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). New York, NY: Routledge. Google Scholar | Crossref | |
|
Hu, J., Liden, R. C. (2013). Relative leader–member exchange within team contexts: How and when social comparison impacts individual effectiveness. Personnel Psychology, 66, 127–172. Google Scholar | Crossref | ISI | |
|
Hülsheger, U. R., Alberts, H. J. E. M., Feinholdt, A., Lang, J. W. B. (2013). Benefits of mindfulness at work: The role of mindfulness in emotion regulation, emotional exhaustion, and job satisfaction. Journal of Applied Psychology, 98, 310–325. Google Scholar | Crossref | Medline | ISI | |
|
Kenny, D. A., Judd, C. M. (1996). A general procedure for the estimation of interdependence. Psychological Bulletin, 119, 138–148. Google Scholar | Crossref | Medline | ISI | |
|
Kenny, D. A., Korchmaros, J. D., Bolger, N. (2003). Lower level mediation in multilevel models. Psychological Methods, 8, 115–128. Google Scholar | Crossref | Medline | ISI | |
|
Kim, E., Bhave, D. P., Glomb, T. M. (2013). Emotion regulation in workgroups: The roles of demographic diversity and relational work context. Personnel Psychology, 66, 613–644. Google Scholar | Crossref | ISI | |
|
LaHuis, D. M., Hartman, M. J., Hakoyama, S., Clark, P. C. (2014). Explained variance measures for multilevel models. Organizational Research Methods, 17, 433–451. Google Scholar | SAGE Journals | ISI | |
|
Leavitt, K., Mitchell, R. R., Peterson, J. (2010). Theory pruning: Strategies to reduce our dense theoretical landscape. Organizational Research Methods, 13, 644–667. Google Scholar | SAGE Journals | ISI | |
|
LeBreton, J. M., Senter, J. L. (2008). Answers to twenty questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815–852. Google Scholar | SAGE Journals | ISI | |
|
Liao, H., Rupp, D. E. (2005). The impact of justice climate and justice orientation on work outcomes: A cross-level multifoci framework. Journal of Applied Psychology, 90, 242–256. Google Scholar | Crossref | Medline | ISI | |
|
Longford, N. (1993). Random coefficient modeling. Oxford, UK: Clarendon. Google Scholar | |
|
Mathieu, J. E., Aguinis, H., Culpepper, S. A., Chen, G. (2012). Understanding and estimating the power to detect cross-level interaction effects in multilevel modeling. Journal of Applied Psychology, 97, 951–966. Google Scholar | Crossref | Medline | ISI | |
|
Mathieu, J., Ahearne, M., Taylor, S. R. (2007). A longitudinal cross-level model of leader and salesperson influences on sales force technology use and performance. Journal of Applied Psychology, 92, 528–537. Google Scholar | Crossref | Medline | ISI | |
|
Raudenbush, S. W., Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage Publications. Google Scholar | |
|
Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15, 351–357. Google Scholar | Crossref | ISI | |
|
Searle, S. R., Casella, G., McCulloch, C. E. (1992). Variance components. New York, NY: Wiley. Google Scholar | Crossref | |
|
Self, S. G., Liang, K. (1987). Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. Journal of the American Statistical Association, 82, 605–610. Google Scholar | Crossref | ISI | |
|
Short, J. C., Ketchen, D. J., Bennett, N., du Toit, M. (2006). An examination of firm, industry, and time effects on performance using random coefficients modeling. Organizational Research Methods, 9, 259–284. Google Scholar | SAGE Journals | ISI | |
|
Snijders, T. A. B., Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). Thousand Oaks, CA: Sage. Google Scholar | |
|
Stram, D., Lee, J.W. (1994). Variance components testing in the longitudinal mixed effects model. Biometrics, 50, 1171–1177. Google Scholar | Crossref | Medline | ISI | |
|
Takeuchi, R., Chen, G., Lepak, D. P. (2009). Through the looking glass of a social system: Cross-level effects of high-performance work systems on employees’ attitudes. Personnel Psychology, 62, 1–29. Google Scholar | Crossref | ISI | |
|
Taylor, P. J., Li, W. D., Shi, K., Borman, W. C. (2008). The transportability of job information across countries. Personnel Psychology, 61, 69–111. Google Scholar | Crossref | ISI | |
|
Uy, M. A., Foo, M. D., Aguinis, H. (2010). Using experience sampling methodology to advance entrepreneurship theory and research. Organizational Research Methods, 13, 31–54. Google Scholar | SAGE Journals | ISI |
