Abstract
Publication bias frequently appears in meta-analyses when the included studies’ results (e.g., p-values) influence the studies’ publication processes. Some unfavorable studies may be suppressed from publication, so the meta-analytic results may be biased toward an artificially favorable direction. Many statistical tests have been proposed to detect publication bias in recent two decades. However, they often make dramatically different assumptions about the cause of publication bias; therefore, they are usually powerful only in certain cases that support their particular assumptions, while their powers may be fairly low in many other cases. Although several simulation studies have been carried out to compare different tests’ powers under various situations, it is typically infeasible to justify the exact mechanism of publication bias in a real-world meta-analysis and thus select the corresponding optimal publication bias test. We introduce a hybrid test for publication bias by synthesizing various tests and incorporating their benefits, so that it maintains relatively high powers across various mechanisms of publication bias. The superior performance of the proposed hybrid test is illustrated using simulation studies and three real-world meta-analyses with different effect sizes. It is compared with many existing methods, including the commonly used regression and rank tests, and the trim-and-fill method.
References
| 1. | Gurevitch, J, Koricheva, J, Nakagawa, S, et al. Meta-analysis and the science of research synthesis. Nature 2018; 555: 175–182. Google Scholar | Crossref | Medline |
| 2. | Turner, EH, Matthews, AM, Linardatos, E, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. New Engl J Med 2008; 358: 252–260. Google Scholar | Crossref | Medline | ISI |
| 3. | Sutton, AJ, Song, F, Gilbody, SM, et al. Modelling publication bias in meta-analysis: a review. Stat Meth Med Res 2000; 9: 421–445. Google Scholar | SAGE Journals | ISI |
| 4. | Hayashino, Y, Noguchi, Y, Fukui, T. Systematic evaluation and comparison of statistical tests for publication bias. J Epidemiol 2005; 15: 235–243. Google Scholar | Crossref | Medline | ISI |
| 5. | Mavridis, D, Salanti, G. Exploring and accounting for publication bias in mental health: a brief overview of methods. Evidence-Based Mental Health 2014; 17: 11–15. Google Scholar | Crossref | Medline |
| 6. | Jin, ZC, Zhou, XH, He, J. Statistical methods for dealing with publication bias in meta-analysis. Stat Med 2015; 34: 343–360. Google Scholar | Crossref | Medline | ISI |
| 7. | Light, RJ, Pillemer, DB. Summing Up: The Science of Reviewing Research. Cambridge, MA: Harvard University Press, 1984. Google Scholar |
| 8. | Sterne, JAC, Egger, M. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol 2001; 54: 1046–1055. Google Scholar | Crossref | Medline | ISI |
| 9. | Begg, CB, Mazumdar, M. Operating characteristics of a rank correlation test for publication bias. Biometrics 1994; 50: 1088–1101. Google Scholar | Crossref | Medline | ISI |
| 10. | Egger, M, Davey Smith, G, Schneider, M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997; 315: 629–634. Google Scholar | Crossref | Medline |
| 11. | Duval, S, Tweedie, R. A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. J Am Stat Assoc 2000; 95: 89–98. Google Scholar | ISI |
| 12. | Macaskill, P, Walter, SD, Irwig, L. A comparison of methods to detect publication bias in meta-analysis. Stat Med 2001; 20: 641–654. Google Scholar | Crossref | Medline | ISI |
| 13. | Peters, JL, Sutton, AJ, Jones, DR, et al. Comparison of two methods to detect publication bias in meta-analysis. JAMA 2006; 295: 676–680. Google Scholar | Crossref | Medline | ISI |
| 14. | Lin, L, Chu, H. Quantifying publication bias in meta-analysis. Biometrics 2018; 74: 785–794. Google Scholar | Crossref | Medline |
| 15. | Peters, JL, Sutton, AJ, Jones, DR, et al. Performance of the trim and fill method in the presence of publication bias and between-study heterogeneity. Stat Med 2007; 26: 4544–4562. Google Scholar | Crossref | Medline | ISI |
| 16. | Sterne, JAC, Gavaghan, D, Egger, M. Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol 2000; 53: 1119–1129. Google Scholar | Crossref | Medline | ISI |
| 17. | Schwarzer, G, Antes, G, Schumacher, M. Inflation of type I error rate in two statistical tests for the detection of publication bias in meta-analyses with binary outcomes. Stat Med 2002; 21: 2465–2477. Google Scholar | Crossref | Medline |
| 18. | Zwetsloot, PP, Van Der Naald, M, Sena, ES, et al. Standardized mean differences cause funnel plot distortion in publication bias assessments. eLife 2017; 6: e24260. Google Scholar | Crossref | Medline |
| 19. | Lin, L. Bias caused by sampling error in meta-analysis with small sample sizes. PLOS ONE 2018; 13: e0204056. Google Scholar | Crossref | Medline |
| 20. | Pustejovsky, JE, Rodgers, MA. Testing for funnel plot asymmetry of standardized mean differences. Res Synth Methods 2019; 10: 57–71. Google Scholar | Crossref | Medline |
| 21. | Lin, L, Chu, H, Murad, MH, et al. Empirical comparison of publication bias tests in meta-analysis. J Gen Intern Med 2018; 33: 1260–1267. Google Scholar | Crossref | Medline |
| 22. | Thompson, SG, Sharp, SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med 1999; 18: 2693–2708. Google Scholar | Crossref | Medline | ISI |
| 23. | Tang, JL, Liu, JLY. Misleading funnel plot for detection of bias in meta-analysis. J Clin Epidemiol 2000; 53: 477–484. Google Scholar | Crossref | Medline |
| 24. | Duval, S, Tweedie, R. Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000; 56: 455–463. Google Scholar | Crossref | Medline | ISI |
| 25. | Shi, L, Lin, L. The trim-and-fill method for publication bias: practical guidelines and recommendations based on a large database of meta-analyses. Medicine 2019; 98: e15987. Google Scholar | Crossref | Medline |
| 26. | Rücker, G, Schwarzer, G, Carpenter, J. Arcsine test for publication bias in meta-analyses with binary outcomes. Stat Med 2008; 27: 746–763. Google Scholar | Crossref | Medline | ISI |
| 27. | Jin, ZC, Wu, C, Zhou, XH, et al. A modified regression method to test publication bias in meta-analyses with binary outcomes. BMC Med Res Methodol 2014; 14: 132. Google Scholar | Crossref | Medline |
| 28. | Harbord, RM, Egger, M, Sterne, JAC. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med 2006; 25: 3443–3457. Google Scholar | Crossref | Medline | ISI |
| 29. | Schwarzer, G, Antes, G, Schumacher, M. A test for publication bias in meta-analysis with sparse binary data. Stat Med 2007; 26: 721–733. Google Scholar | Crossref | Medline |
| 30. | Conneely, KN, Boehnke, M. So many correlated tests, so little time! Rapid adjustment of P values for multiple correlated tests. Am J Hum Genet 2007; 81: 1158–1168. Google Scholar | Crossref | Medline | ISI |
| 31. | Pan, W, Kim, J, Zhang, Y, et al. A powerful and adaptive association test for rare variants. Genetics 2014; 210: 1081–1095. Google Scholar | Crossref |
| 32. | Xu, G, Lin, L, Wei, P, et al. An adaptive two-sample test for high-dimensional means. Biometrika 2016; 103: 609–624. Google Scholar | Crossref | Medline |
| 33. | Efron, B, Tibshirani, RJ. An Introduction to the Bootstrap. Boca Raton, FL: CRC Press, 1998. Google Scholar |
| 34. | Boos, DD, Zhang, J. Monte Carlo evaluation of resampling-based hypothesis tests. J Am Stat Assoc 2000; 95: 486–492. Google Scholar | Crossref | ISI |
| 35. | Takkouche, B, Cadarso-Suárez, C, Spiegelman, D. Evaluation of old and new tests of heterogeneity in epidemiologic meta-analysis. Am J Epidemiol 1999; 150: 206–215. Google Scholar | Crossref | Medline | ISI |
| 36. | Higgins, JPT. Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified. Int J Epidemiol 2008; 37: 1158–1160. Google Scholar | Crossref | Medline | ISI |
| 37. | Adams, DC, Gurevitch, J, Rosenberg, MS. Resampling tests for meta-analysis of ecological data. Ecology 1997; 78: 1277–1283 Google Scholar | Crossref |
| 38. | Higgins, JPT, Thompson, SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002; 21: 1539–1558. Google Scholar | Crossref | Medline | ISI |
| 39. | Demetrashvili, N, Van den Heuvel, ER. Confidence intervals for intraclass correlation coefficients in a nonlinear dose–response meta-analysis. Biometrics 2015; 71: 548–555. Google Scholar | Crossref | Medline |
| 40. | Hedges, LV, Olkin, I. Statistical Methods for Meta-Analysis. Orlando, FL: Academic Press, 1985. Google Scholar | Crossref |
| 41. | Plourde, G, Pancholy, SB, Nolan, J, et al. Radiation exposure in relation to the arterial access site used for diagnostic coronary angiography and percutaneous coronary intervention: a systematic review and meta-analysis. Lancet 2015; 386: 2192–2203. Google Scholar | Crossref | Medline | ISI |
| 42. | Paige, NM, Miake-Lye, IM, Booth, MS, et al. Association of spinal manipulative therapy with clinical benefit and harm for acute low back pain: systematic review and meta-analysis. JAMA 2017; 317: 1451–1460. Google Scholar | Crossref | Medline |
| 43. | Whiting, PF, Wolff, RF, Deshpande, S, et al. Cannabinoids for medical use: a systematic review and meta-analysis. JAMA 2015; 313: 2456–2473. Google Scholar | Crossref | Medline |
| 44. | Sterne, JAC, Sutton, AJ, Ioannidis, JPA, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ 2011; 343: d4002. Google Scholar | Crossref | Medline |
| 45. | Peters, JL, Sutton, AJ, Jones, DR, et al. Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J Clin Epidemiol 2008; 61: 991–996. Google Scholar | Crossref | Medline | ISI |
| 46. | Lin, L. Graphical augmentations to sample-size-based funnel plot in meta-analysis. Res Synth Methods 2019; 10: 376–388. Google Scholar | Crossref | Medline |
| 47. | Schwarzer, G, Carpenter, J, Rücker, G. Empirical evaluation suggests Copas selection model preferable to trim-and-fill method for selection bias in meta-analysis. J Clin Epidemiol 2010; 63: 282–288. Google Scholar | Crossref | Medline |
| 48. | Hedges, LV. Modeling publication selection effects in meta-analysis. Stat Sci 1992; 7: 246–255. Google Scholar | Crossref |
| 49. | Silliman, NP. Hierarchical selection models with applications in meta-analysis. J Am Stat Assoc 1997; 92: 926–936. Google Scholar | Crossref |
| 50. | Copas, JB, Shi, JQ. A sensitivity analysis for publication bias in systematic reviews. Stat Meth Med Res 2001; 10: 251–265. Google Scholar | SAGE Journals | ISI |
| 51. | Copas, J, Dwan, K, Kirkham, J, et al. A model-based correction for outcome reporting bias in meta-analysis. Biostatistics 2013; 15: 370–383. Google Scholar | Crossref | Medline |

