![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Context effects, where a characteristic of an upper-level unit or cluster (e.g., a country) affects outcomes and relationships at a lower level (e.g., that of the individual), are a primary object of sociological inquiry. In recent years, sociologists have increasingly analyzed such effects using quantitative multilevel modeling. Our review of multilevel studies in leading sociology journals shows that most assume the effects of lower-level control variables to be invariant across clusters, an assumption that is often implausible. Comparing mixed-effects (random-intercept and slope) models, cluster-robust pooled OLS, and two-step approaches, we find that erroneously assuming invariant coefficients reduces the precision of estimated context effects. Semi-formal reasoning and Monte Carlo simulations indicate that loss of precision is largest when there is pronounced cross-cluster heterogeneity in the magnitude of coefficients, when there are marked compositional differences among clusters, and when the number of clusters is small. Although these findings suggest that practitioners should fit more flexible models, illustrative analyses of European Social Survey data indicate that maximally flexible mixed-effects models do not perform well in real-life settings. We discuss the need to balance parsimony and flexibility, and we demonstrate the encouraging performance of one prominent approach for reducing model complexity.
References
|
Bates, Douglas, Kliegl, Reinhold, Vasishth, Shravan, Baayen, Harald. 2015. “Parsimonious Mixed Models.” arXiv preprint arXiv:1506.04967. Retrieved April 22, 2016 (http://arxiv.org/abs/1506.04967). Google Scholar | |
|
Bates, Douglas, Mächler, Martin, Bolker, Ben, Walker, Steve. 2015. “Fitting Linear Mixed-Effects Models Using lme4.” Journal of Statistical Software 67(1):1–48. Google Scholar | Crossref | ISI | |
|
Bryan, Mark L., Jenkins, Stephen P. 2016. “Multilevel Modelling of Country Effects: A Cautionary Tale.” European Sociological Review 32(1):3–22. Google Scholar | Crossref | ISI | |
|
Cameron, A. Colin, Miller, Douglas L. 2015. “A Practitioner’s Guide to Cluster-Robust Inference.” Journal of Human Resources 50(2):317–72. Google Scholar | Crossref | ISI | |
|
Durkheim, Emile . 1897. Suicide: A Study in Sociology. Glencoe, IL: Free Press. Google Scholar | |
|
Elff, Martin, Heisig, Jan P., Schaeffer, Merlin, Shikano, Susumu. 2016. “No Need to Turn Bayesian in Multilevel Analysis with Few Clusters: How Frequentist Methods Provide Unbiased Estimates and Accurate Inference.” SocArXiv/Open Science Framework (Version 2, December 10, 2016; https://osf.io/preprints/socarxiv/z65s4/). Google Scholar | |
|
Enders, Craig K., Tofighi, Davood. 2007. “Centering Predictor Variables in Cross-Sectional Multilevel Models: A New Look at an Old Issue.” Psychological Methods 12(2):121–38. Google Scholar | Crossref | Medline | ISI | |
|
Esarey, Justin, Menger., Andrew 2015. “Practical and Effective Approaches to Dealing with Clustered Data.” Presented at the 2015 Annual Meeting of the Society for Political Methodology at the University of Rochester, Rochester, New York. Google Scholar | |
|
European Social Survey (ESS) Round 6 . 2016. ESS-6 2012 Documentation Report, ed. 2.2. Bergen: European Social Survey Data Archive, NSD – Norwegian Centre for Research Data for ESS ERIC. Google Scholar | |
|
Gelman, Andrew, Hill, Jennifer. 2007. Data Analysis Using Regression and Multilevel/Hierarchichal Models. Cambridge, UK: Cambridge University Press. Google Scholar | |
|
Goldstein, Harvey . 2011. “Bootstrapping in Multilevel Models.” Pp. 163–71 in Handbook of Advanced Multilevel Analysis, edited by Hox, J. J., Roberts, J. K. New York: Routledge. Google Scholar | |
|
Imbens, Guido W., Kolesár, Michal. 2016. “Robust Standard Errors in Small Samples: Some Practical Advice.” Review of Economics and Statistics 98(4):701–712. Google Scholar | Crossref | |
|
Joe, Harry . 2006. “Generating Random Correlation Matrices Based on Partial Correlations.” Journal of Multivariate Analysis 97(10):2177–89. Google Scholar | Crossref | |
|
Kézdi, Gabor . 2004. “Robust Standard Error Estimation in Fixed-Effects Panel Models.” Hungarian Statistical Review 89(9):96–116. Google Scholar | |
|
Kloek, Teun . 1981. “OLS Estimation in a Model Where a Microvariable Is Explained by Aggregates and Contemporaneous Disturbances Are Equicorrelated.” Econometrica 49(1):205–207. Google Scholar | Crossref | |
|
Lewis, Jeffrey B., Linzer, Drew A. 2005. “Estimating Regression Models in Which the Dependent Variable Is Based on Estimates.” Political Analysis 13(4):345–64. Google Scholar | Crossref | ISI | |
|
Lindley, Dennis V., Smith, Adrian F. M. 1972. “Bayes Estimates for the Linear Model.” Journal of the Royal Statistical Society. Series B (Methodological) 34(1):1–41. Google Scholar | |
|
Maas, Cora J. M., Hox, Joop J. 2004. “The Influence of Violations of Assumptions on Multilevel Parameter Estimates and Their Standard Errors.” Computational Statistics & Data Analysis 46(3):427–40. Google Scholar | Crossref | ISI | |
|
MacKinnon, James G., Webb, Matthew D. 2014. “Wild Bootstrap Inference for Wildly Different Cluster Sizes.” Working paper, Queen’s Economics Department, Queen’s University, Kingston, Canada. Retrieved June 26, 2015 (http://www.econstor.eu/handle/10419/97471). Google Scholar | |
|
Moulton, Brent R. 1986. “Random Group Effects and the Precision of Regression Estimates.” Journal of Econometrics 32(3):385–97. Google Scholar | Crossref | ISI | |
|
Moulton, Brent R. 1990. “An Illustration of a Pitfall in Estimating the Effects of Aggregate Variables on Micro Units.” Review of Economics and Statistics 72(2):334–38. Google Scholar | Crossref | ISI | |
|
Müller, Samuel, Scealy, J. L., Welsh, A. H. 2013. “Model Selection in Linear Mixed Models.” Statistical Science 28(2):135–67. Google Scholar | Crossref | |
|
Raudenbush, Stephen W., Bryk, Anthony S. 2002. Hierarchical Linear Models: Applications and Data Analysis Methods, 2nd ed. Thousand Oaks, CA: Sage Publications. Google Scholar | |
|
Rogers, William . 1993. “Regression Standard Errors in Clustered Samples.” Stata Technical Bulletin 3(13):19–23. Google Scholar | |
|
Sampson, Robert J. 2013. Great American City: Chicago and the Enduring Neighborhood Effect. Chicago: University of Chicago Press. Google Scholar | |
|
Schmidt-Catran, Alexander W., Fairbrother, Malcolm. 2016. “The Random Effects in Multilevel Models: Getting Them Wrong and Getting Them Right.” European Sociological Review 32(1):23–38. Google Scholar | Crossref | ISI | |
|
Stegmueller, Daniel . 2013. “How Many Countries for Multilevel Modeling? A Comparison of Frequentist and Bayesian Approaches.” American Journal of Political Science 57(3):748–61. Google Scholar | Crossref | ISI | |
|
te Grotenhuis, Manfred, Pelzer, Ben, Eisinga, Rob, Nieuwenhuis, Rense, Schmidt-Catran, Alexander, Konig, Ruben. 2016. “When Size Matters: Advantages of Weighted Effect Coding in Observational Studies.” International Journal of Public Health 62(1):163–67. Google Scholar | Crossref | Medline | |
|
United Nations Development Programme , ed. 2015. Human Development Report 2015. New York: United Nations Development Programme. Google Scholar | Crossref | |
|
White, Halbert . 1980. “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity.” Econometrica 48(4):817–38. Google Scholar | Crossref | ISI | |
|
Williams, Rick L. 2000. “A Note on Robust Variance Estimation for Cluster-Correlated Data.” Biometrics 56(2):645–46. Google Scholar | Crossref | Medline | ISI | |
|
Wooldridge, Jeffrey M. 2003. “Cluster-Sample Methods in Applied Econometrics.” American Economic Review 93(2):133–38. Google Scholar | Crossref | ISI | |
|
Wooldridge, Jeffrey M. 2014. Introduction to Econometrics. Europe, Middle East and Africa Edition. Andover, MA: Cengage Learning. Google Scholar |
