Abstract
Abstract. Currently, dedicated graphical displays to depict study-level statistical power in the context of meta-analysis are unavailable. Here, we introduce the sunset (power-enhanced) funnel plot to visualize this relevant information for assessing the credibility, or evidential value, of a set of studies. The sunset funnel plot highlights the statistical power of primary studies to detect an underlying true effect of interest in the well-known funnel display with color-coded power regions and a second power axis. This graphical display allows meta-analysts to incorporate power considerations into classic funnel plot assessments of small-study effects. Nominally significant, but low-powered, studies might be seen as less credible and as more likely being affected by selective reporting. We exemplify the application of the sunset funnel plot with two published meta-analyses from medicine and psychology. Software to create this variation of the funnel plot is provided via a tailored R function. In conclusion, the sunset (power-enhanced) funnel plot is a novel and useful graphical display to critically examine and to present study-level power in the context of meta-analysis.
References
1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088–1101. https://doi.org/10.2307/2533446
(2015). Contour plot assessment of existing meta-analyses confirms robust association of statin use and acute kidney injury risk. Journal of Clinical Epidemiology, 68, 1138–1143. https://doi.org/10.1016/j.jclinepi.2015.05.030
(2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56, 455–463. https://doi.org/10.1111/j.0006-341X.2000.00455.x
(1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629–634. https://doi.org/10.1136/bmj.315.7109.629
(2017). Detecting and avoiding likely false-positive finding: A practical guide. Biological Reviews, 92, 1941–1968. https://doi.org/10.1111/brv.12315
(2017). The power of bias in economics research. The Economic Journal, 127, F236–F265. https://doi.org/10.1111/ecoj.12461
(2007). An exploratory test for an excess of significant findings. Clinical Trials, 4, 245–253. https://doi.org/10.1177/1740774507079441
(2018). metaviz. [R software package]. Retrieved from https://CRAN.R-project.org/package=metaviz
(2019). Visual inference for the funnel plot in meta-analysis. Zeitschrift für Psychologie, 227, 83–89. https://doi.org/10.1027/2151-2604/a000358
(2012). Graphical augmentations to the funnel plot assess the impact of additional evidence on a meta-analysis. Journal of Clinical Epidemiology, 65, 511–519. https://doi.org/10.1016/j.jclinepi.2011.10.009
(2006). Evidence based medicine: The case of the misleading funnel plot. British Medical Journal, 333, 597–600. https://doi.org/10.1136/bmj.333.7568.597
(1984). Summing up: The science of reviewing research. Cambridge, MA: Harvard University Press.
(2017). Randomised, double-blind, placebo-controlled trials of non-individualised homeopathic treatment: Systematic review and meta-analysis. Systematic Reviews, 6, 63. https://doi.org/10.1186/s13643-017-0445-3
(2003). Meta-analysis and power: Some suggestions for the use of power in research synthesis. Understanding Statistics, 2, 1–12. https://doi.org/10.1207/S15328031US0201_01
(2008). Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. Journal of Clinical Epidemiology, 61, 991–996. https://doi.org/10.1016/j.jclinepi.2007.11.010
(2015). Meta-analysis of associations between human brain volume and intelligence differences: How strong are they and what do they mean? Neuroscience & Biobehavioral Reviews, 57, 411–432. https://doi.org/10.1016/j.neubiorev.2015.09.017
(2018). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/
. (2019). How to detect publication bias in psychological research: A comparative evaluation of six statistical methods. Zeitschrift für Psychologie, 227, 261–279. https://doi.org/10.1027/2151-2604/a000386
(2008). Arcsine test for publication bias in meta-analyses with binary outcomes. Statistics in Medicine, 27, 746–763. https://doi.org/10.1002/sim.2971
(2013). Less is less: A systematic review of graph use in meta‐analyses. Research Synthesis Methods, 4, 209–219. https://doi.org/10.1002/jrsm.1076
(2016). The replicability-index: Quantifying statistical research integrity. Retrieved from https://replicationindex.wordpress.com/2016/01/31/a-revised-introduction-to-the-r-index/
(2015). Quantifying the risk of error when interpreting funnel plots. Systematic Reviews, 4, 24. https://doi.org/10.1186/s13643-015-0004-8
(2001). Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. Journal of Clinical Epidemiology, 54, 1046–1055. https://doi.org/10.1016/S0895-4356(01)00377-8
(2000). Publication and related bias in meta-analysis: Power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology, 53, 1119–1129. https://doi.org/10.1016/S0895-4356(00)00242-0
(2011). Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. British Medical Journal, 343, d4002. https://doi.org/10.1136/bmj.d4002
(2017). Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLoS Biology, 15, e2000797. https://doi.org/10.1371/journal.pbio.2000797
(2005). In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. Journal of Clinical Epidemiology, 58, 894–901. https://doi.org/10.1016/j.jclinepi.2005.01.006
(2016). ggplot2: Elegant graphics for data analysis. Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-24277-4
(