![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
When a multisite randomized trial reveals between-site variation in program impact, methods are needed for further investigating heterogeneous mediation mechanisms across the sites. We conceptualize and identify a joint distribution of site-specific direct and indirect effects under the potential outcomes framework. A method-of-moments procedure incorporating ratio-of-mediator-probability weighting (RMPW) consistently estimates the causal parameters. This strategy conveniently relaxes the assumption of no Treatment × Mediator interaction while greatly simplifying the outcome model specification without invoking strong distributional assumptions. We derive asymptotic standard errors that reflect the sampling variability of the estimated weight. We also offer an easy-to-use R package, MultisiteMediation, that implements the proposed method. It is freely available at the Comprehensive R Archive Network (http://cran.r-project.org/web/packages/MultisiteMediation).
References
|
Albert, J. M., Nelson, S. (2011). Generalized causal mediation analysis. Biometrics, 67, 1028–1038. Google Scholar | Crossref | Medline | |
|
Avin, C., Shpitser, I., Pearl, J. (2005). Identifiability of path-specific effects. Los Angeles: Department of Statistics, UCLA. Google Scholar | |
|
Bates, D., Maechler, M., Bolker, B., Walker, S. (2014). lme4: Linear mixed-effects models using Eigen and s4 (R package version 1.1-7). Retrievable from https://cran.r-project.org/web/packages/lme4/index.html. Google Scholar | |
|
Bauer, D. J., Preacher, K. J., Gil, K. M. (2006). Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: New procedures and recommendations. Psychological Methods, 11, 142. Google Scholar | Crossref | Medline | |
|
Bein, E., Deutsch, J., Porter, K., Qin, X., Yang, C., Hong, G. (2015). Technical report on two-step estimation in RMPW analysis. Oakland, CA: MDRC. Google Scholar | |
|
Bind, M.-A., Vanderweele, T., Coull, B., Schwartz, J. (2016). Causal mediation analysis for longitudinal data with exogenous exposure. Biostatistics, 17, 122–134. Google Scholar | Medline | |
|
Bloom, H., Hill, C. J., Riccio, J. (2005). Modeling cross-site experimental differences to find out why program effectiveness varies. In Bloom, H. S. (Ed.), Learning more from social experiments: Evolving analytic approaches (pp. 37–74). New York, NY: Russell Sage Foundation. Google Scholar | |
|
Diggle, P., Heagerty, P., Liang, K.-Y., Zeger, S. (2002). Analysis of longitudinal data. Oxford, England: Oxford University Press. Google Scholar | |
|
Fitzmaurice, G. M., Lipsitz, S. R., Ibrahim, J. G. (2007). A note on permutation tests for variance components in multilevel generalized linear mixed models. Biometrics, 63, 942–946. Google Scholar | Crossref | Medline | |
|
Flores, C. A., Flores-Lagunes, A. (2013). Partial identification of local average treatment effects with an invalid instrument. Journal of Business & Economic Statistics, 31, 534–545. Google Scholar | Crossref | |
|
Goldstein, H. (2011). Multilevel statistical models (Vol. 922). Chichester, England: John Wiley. Google Scholar | |
|
Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators. Econometrica: Journal of the Econometric Society, 50, 1029–1054. Google Scholar | Crossref | |
|
Hedeker, D., Gibbons, R. D. (2006). Longitudinal data analysis (Vol. 451). Hoboken, NJ: John Wiley. Google Scholar | |
|
Hirano, K., Imbens, G. W. (2001). Estimation of causal effects using propensity score weighting: An application to data on right heart catheterization. Health Services and Outcomes Research Methodology, 2, 259–278. Google Scholar | Crossref | |
|
Hong, G. (2010). Ratio of mediator probability weighting for estimating natural direct and indirect effects. Proceedings of the American Statistical Association, Biometrics Section (pp. 2401–2415). Alexandria, VA: American Statistical Association. Google Scholar | |
|
Hong, G. (2015). Causality in a social world: Moderation, mediation and spill-over. West Sussex, England: John Wiley. Google Scholar | Crossref | |
|
Hong, G., Deutsch, J., Hill, H. D. (2011). Parametric and non-parametric weighting methods for estimating mediation effects: An application to the national evaluation of welfare-to-work strategies. In Proceedings of the American Statistical Association, Social Statistics Section (pp. 3215–3229). Alexandria, VA: American Statistical Association. Google Scholar | |
|
Hong, G., Deutsch, J., Hill, H. D. (2015). Ratio-of-mediator-probability weighting for causal mediation analysis in the presence of treatment-by-mediator interaction. Journal of Educational and Behavioral Statistics, 40, 307–340. Google Scholar | SAGE Journals | |
|
Hong, G., Nomi, T. (2012). Weighting methods for assessing policy effects mediated by peer change. Journal of Research on Educational Effectiveness, 5, 261–289. Google Scholar | Crossref | |
|
Hong, G., Raudenbush, S. W. (2006). Evaluating kindergarten retention policy: A case study of causal inference for multilevel observational data. Journal of the American Statistical Association, 101, 901–910. Google Scholar | Crossref | |
|
Horvitz, D. G., Thompson, D. J. (1952). A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47, 663–685. Google Scholar | Crossref | |
|
Huber, M. (2014). Identifying causal mechanisms (primarily) based on inverse probability weighting. Journal of Applied Econometrics, 29, 920–943. Google Scholar | Crossref | |
|
Hudgens, M. G., Halloran, M. E. (2008). Toward causal inference with interference. Journal of the American Statistical Association, 103, 832–842. Google Scholar | Crossref | Medline | |
|
Imai, K., Keele, L., Tingley, D. (2010). A general approach to causal mediation analysis. Psychological Methods, 15, 309. Google Scholar | Crossref | Medline | |
|
Imai, K., Keele, L., Yamamoto, T. (2010). Identification, inference and sensitivity analysis for causal mediation effects. Statistical Science, 25, 51–71. Google Scholar | Crossref | |
|
Imai, K., Yamamoto, T. (2013). Identification and sensitivity analysis for multiple causal mechanisms: Revisiting evidence from framing experiments. Political Analysis, 21, 141–171. Google Scholar | Crossref | |
|
Johnson, T., Gritz, M., Jackson, R., Burghardt, J., Boussy, C., Leonard, J., Orians, C. (1999). National job corps study: Report on the process analysis (Research and Evaluation Report Series 8140-510). Princeton, NJ: Mathematica Policy Research. Google Scholar | |
|
Judd, C. M., Kenny, D. A. (1981). Process analysis estimating mediation in treatment evaluations. Evaluation Review, 5, 602–619. Google Scholar | SAGE Journals | |
|
Kenny, D. A., Korchmaros, J. D., Bolger, N. (2003). Lower level mediation in multilevel models. Psychological Methods, 8, 115. Google Scholar | Crossref | Medline | |
|
Kling, J. R., Liebman, J. B., Katz, L. F. (2007). Experimental analysis of neighborhood effects. Econometrica, 75, 83–119. Google Scholar | Crossref | |
|
Krull, J. L., MacKinnon, D. P. (2001). Multilevel modeling of individual and group level mediated effects. Multivariate Behavioral Research, 36, 249–277. Google Scholar | Crossref | Medline | |
|
Lange, T., Rasmussen, M., Thygesen, L. (2014). Assessing natural direct and indirect effects through multiple pathways. American Journal of Epidemiology, 179, 513. Google Scholar | Crossref | Medline | |
|
Lange, T., Vansteelandt, S., Bekaert, M. (2012). A simple unified approach for estimating natural direct and indirect effects. American Journal of Epidemiology, 176, 190–195. Google Scholar | Crossref | Medline | |
|
Leite, W. L., Jimenez, F., Kaya, Y., Stapleton, L. M., MacInnes, J. W., Sandbach, R. (2015). An evaluation of weighting methods based on propensity scores to reduce selection bias in multilevel observational studies. Multivariate Behavioral Research, 50, 265–284. Google Scholar | Crossref | Medline | |
|
MacKinnon, D. P., Dwyer, J. H. (1993). Estimating mediated effects in prevention studies. Evaluation Review, 17, 144–158. Google Scholar | SAGE Journals | |
|
Newey, W. K. (1984). A method of moments interpretation of sequential estimators. Economics Letters, 14, 201–206. Google Scholar | Crossref | |
|
Neyman, J., Iwaszkiewicz, K. (1935). Statistical problems in agricultural experimentation. Supplement to the Journal of the Royal Statistical Society, 2, 107–180. Google Scholar | Crossref | |
|
Pearl, J. (2001). Direct and indirect effects. In Breese, J., Koller, D. (Eds.), Proceedings of the seventeenth conference on uncertainty in artificial intelligence (pp. 411–420). San Francisco, CA: Morgan Kaufmann. Google Scholar | |
|
Preacher, K. J., Zyphur, M. J., Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15, 209. Google Scholar | Crossref | Medline | |
|
Raudenbush, S. W., Bloom, H. (2015). Using multi-site randomized trials to learn about and from a distribution of program impacts. American Journal of Evaluation, 36, 475–499. Google Scholar | SAGE Journals | |
|
Raudenbush, S. W., Reardon, S. F., Nomi, T. (2012). Statistical analysis for multisite trials using instrumental variables with random coefficients. Journal of Research on Educational Effectiveness, 5, 303–332. Google Scholar | Crossref | |
|
Reardon, S. F., Raudenbush, S. W. (2013). Under what assumptions do site-by-treatment instruments identify average causal effects? Sociological Methods & Research. doi:10.1177/0049124113494575 Google Scholar | SAGE Journals | |
|
Robins, J. M. (2000). Marginal structural models versus structural nested models as tools for causal inference. In Halloran, M. E., Berry, D. (Eds.), Statistical models in epidemiology, the environment, and clinical trials (pp. 95–133). New York, NY: Springer. Google Scholar | Crossref | |
|
Robins, J. M., Greenland, S. (1992). Identifiability and exchangeability for direct and indirect effects. Epidemiology, 3, 143–155. Google Scholar | Crossref | Medline | |
|
Rosenbaum, P. R. (1987). Model-based direct adjustment. Journal of the American Statistical Association, 82, 387–394. Google Scholar | Crossref | |
|
Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. The Annals of Statistics, 6, 34–58. Google Scholar | Crossref | |
|
Rubin, D. B. (1980). Randomization analysis of experimental data: The fisher randomization test comment. Journal of the American Statistical Association, 75, 591–593. Google Scholar | |
|
Rubin, D. B. (1986). Statistics and causal inference: Comment: Which ifs have causal answers. Journal of the American Statistical Association, 81, 961–962. Google Scholar | |
|
Rubin, D. B. (1990). Formal mode of statistical inference for causal effects. Journal of Statistical Planning and Inference, 25, 279–292. Google Scholar | Crossref | |
|
Seltzer, J. A. (1994). Consequences of marital dissolution for children. Annual Review of Sociology, 20, 235–266. Google Scholar | Crossref | Medline | |
|
Spybrook, J., Raudenbush, S. W. (2009). An examination of the precision and technical accuracy of the first wave of group-randomized trials funded by the institute of education sciences. Educational Evaluation and Policy Analysis, 31, 298–318. Google Scholar | SAGE Journals | |
|
Tchetgen Tchetgen, E. J. (2013). Inverse odds ratio-weighted estimation for causal mediation analysis. Statistics in Medicine, 32, 4567–4580. Google Scholar | Crossref | Medline | |
|
Tchetgen Tchetgen, E. J., Shpitser, I. (2012). Semiparametric theory for causal mediation analysis: Efficiency bounds, multiple robustness, and sensitivity analysis. Annals of Statistics, 40, 1816. Google Scholar | Crossref | Medline | |
|
VanderWeele, T. J. (2010a). Bias formulas for sensitivity analysis for direct and indirect effects. Epidemiology (Cambridge, Mass.), 21, 540. Google Scholar | Crossref | Medline | |
|
VanderWeele, T. J. (2010b). Direct and indirect effects for neighborhood-based clustered and longitudinal data. Sociological Methods & Research, 38, 515–544. Google Scholar | SAGE Journals | |
|
VanderWeele, T. J., Chiba, Y. (2014). Sensitivity analysis for direct and indirect effects in the presence of exposure-induced mediator–outcome confounders. Epidemiology, Biostatistics, and Public Health, 11, 1–16. Google Scholar | |
|
Vanderweele, T. J., Hong, G., Jones, S. M., Brown, J. L. (2013). Mediation and spillover effects in group-randomized trials: A case study of the 4rs educational intervention. Journal of the American Statistical Association, 108, 469–482. Google Scholar | Crossref | Medline | |
|
VanderWeele, T. J., Vansteelandt, S. (2010). Odds ratios for mediation analysis for a dichotomous outcome. American Journal of Epidemiology, 172, 1339–1348. Google Scholar | Crossref | Medline | |
|
Weiss, M. J., Bloom, H. S., Brock, T. (2014). A conceptual framework for studying the sources of variation in program effects. Journal of Policy Analysis and Management, 33, 778–808. Google Scholar | Crossref | |
|
Zhang, Z., Zyphur, M. J., Preacher, K. J. (2009). Testing multilevel mediation using hierarchical linear models problems and solutions. Organizational Research Methods, 12, 695–719. Google Scholar | SAGE Journals |
