Funnel plot

<h2 id="quotation">Quotation</h2>
<p>Funnel plots, introduced by Light and Pillemer in 1984<a class="footnote-ref" id="fnref:1" href="#fn:1"><sup>1</sup></a>
and discussed in detail by <a href="/facts/Matthias_Egger/NfKs0hCY">Matthias Egger</a> and colleagues,<a class="footnote-ref" id="fnref:2" href="#fn:2"><sup>2</sup></a><a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a>
are useful adjuncts to meta-analyses. A funnel plot is a <a href="/facts/Scatterplot/n03xBtBL">scatterplot</a> of treatment effect against a measure of study precision. It is used primarily as a visual aid for detecting bias or systematic <a href="/facts/Study_heterogeneity/AnKsu2K3">heterogeneity</a>. A <a href="/facts/Symmetry/hpu7DK8p">symmetric</a> inverted funnel shape arises from a ‘well-behaved’ data set, in which publication bias is unlikely. An asymmetric funnel indicates a relationship between treatment effect estimate and study precision. This suggests the possibility of either <a href="/facts/Publication_bias/Wbmr1WjW">publication bias</a> or a systematic difference between studies of higher and lower precision (typically ‘small study effects’). Asymmetry can also arise from use of an inappropriate effect measure. Whatever the cause, an asymmetric funnel plot leads to doubts over the appropriateness of a simple meta-analysis and suggests that there needs to be investigation of possible causes.
</p><p>A variety of choices of measures of ‘study precision’ is available, including total sample size, <a href="/facts/Standard_error_(statistics)/Iv96AyCD">standard error</a> of the treatment effect, and inverse <a href="/facts/Variance/ULBJKXD1">variance</a> of the treatment effect (<a href="/facts/Weight_function/YNJL7o99">weight</a>). Sterne and Egger have compared these with others, and conclude that the standard error is to be recommended.<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a>
When the standard error is used, straight lines may be drawn to define a region within which 95% of points might lie in the absence of both <a href="/facts/Study_heterogeneity/AnKsu2K3">heterogeneity</a> and publication bias.<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a>
</p><p>In common with <a href="/facts/Confidence_interval/NS8mG5UE">confidence interval</a> plots, funnel plots are conventionally drawn with the treatment effect measure on the <a href="/facts/Cartesian_coordinate_system/lkTqvMmB">horizontal axis</a>, so that study precision appears on the vertical axis, breaking with the general rule. Since funnel plots are principally visual aids for detecting asymmetry along the treatment effect axis, this makes them considerably easier to interpret.
</p>
<h2 id="criticism">Criticism</h2>
<p>The funnel plot is not without problems. 
If high-precision studies are different from low-precision studies with respect to <a href="/facts/Effect_size/mIpNrPte">effect size</a> (e.g., due to different populations examined) a funnel plot may give a wrong impression of publication bias.<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a>
The appearance of the funnel plot can change quite dramatically depending on the scale on the y-axis — whether it is the inverse square error or the trial size.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> Researchers have a poor ability to visually discern publication bias from funnel plots.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a>
</p>
<h2 id="see-also">See also</h2>
<ul><li><a href="/facts/Galbraith_plot/AGS0V9wm">Galbraith plot</a></li>
<li><a href="/facts/Forest_plot/ENCSLyaq">Forest plot</a></li>
<li><a href="/facts/Systematic_review/dGvq3BGx">Systematic review</a></li></ul>

<h2 id="further-reading">Further reading</h2>
<ul><li>Sterne, J. A. C.; Sutton, A. J.; Ioannidis, J. P. A.; et al. (2011), "Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials", <i>BMJ</i>, 343: d4002, <a href="/facts/Doi_(identifier)/muM9Etpq">doi</a>:<a href="https://doi.org/10.1136%2Fbmj.d4002">10.1136/bmj.d4002</a>, <a href="/facts/PMID_(identifier)/JlHAvMHt">PMID</a> <a href="https://pubmed.ncbi.nlm.nih.gov/21784880">21784880</a></li>
<li>Higgins, J.P.T.; Thomas, J.; Chandler, J.; Cumpston, M.; Li, T.; Page, M.J.; Welch, V.A. (2019), <a href="http://www.handbook.cochrane.org/"><i>Cochrane handbook for systematic reviews of interventions</i></a> (2nd ed.), Wiley Blackwell, <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 9781119536611</li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1"><p>R. J. Light; D. B. Pillemer (1984). Summing up: The Science of Reviewing Research. Cambridge, Massachusetts.: Harvard University Press. ISBN 978-0-674-85431-4. <a href="978-0-674-85431-4" target="_blank">978-0-674-85431-4</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li>
<li id="fn:2"><p>Matthias Egger, G. Davey Smith, M. Schneider & C. Minder (September 1997). "Bias in meta-analysis detected by a simple, graphical test". BMJ. 315 (7109): 629–634. doi:10.1136/bmj.315.7109.629. PMC 2127453. PMID 9310563.{{cite journal}}:  CS1 maint: multiple names: authors list (link) <a href="/wiki/Matthias_Egger" target="_blank">/wiki/Matthias_Egger</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li>
<li id="fn:3"><p>Jonathan A. C. Sterne; Matthias Egger (October 2001). "Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis". Journal of Clinical Epidemiology. 54 (10): 1046–55. doi:10.1016/S0895-4356(01)00377-8. PMID 11576817. <a href="/wiki/Journal_of_Clinical_Epidemiology" target="_blank">/wiki/Journal_of_Clinical_Epidemiology</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li>
<li id="fn:4"><p>Jonathan A. C. Sterne; Matthias Egger (October 2001). "Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis". Journal of Clinical Epidemiology. 54 (10): 1046–55. doi:10.1016/S0895-4356(01)00377-8. PMID 11576817. <a href="/wiki/Journal_of_Clinical_Epidemiology" target="_blank">/wiki/Journal_of_Clinical_Epidemiology</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li>
<li id="fn:5"><p>Jonathan A. C. Sterne; Matthias Egger (October 2001). "Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis". Journal of Clinical Epidemiology. 54 (10): 1046–55. doi:10.1016/S0895-4356(01)00377-8. PMID 11576817. <a href="/wiki/Journal_of_Clinical_Epidemiology" target="_blank">/wiki/Journal_of_Clinical_Epidemiology</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li>
<li id="fn:6"><p>Joseph Lau, John P. A. Ioannidis, Norma Terrin, Christopher H. Schmid & Ingram Olkin (September 2006). "The case of the misleading funnel plot". BMJ. 333 (7568): 597–600. doi:10.1136/bmj.333.7568.597. PMC 1570006. PMID 16974018.{{cite journal}}:  CS1 maint: multiple names: authors list (link) <a href="/wiki/John_P._A._Ioannidis" target="_blank">/wiki/John_P._A._Ioannidis</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li>
<li id="fn:7"><p>Jin-Ling Tang; Joseph LY Liu (May 2000). "Misleading funnel plot for detection of bias in meta-analysis". Journal of Clinical Epidemiology. 53 (5): 477–484. doi:10.1016/S0895-4356(99)00204-8. PMID 10812319. <a href="/wiki/Journal_of_Clinical_Epidemiology" target="_blank">/wiki/Journal_of_Clinical_Epidemiology</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li>
<li id="fn:8"><p>Terrin, N.; Schmid, C. H.; Lau, J. (2005). "In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias". Journal of Clinical Epidemiology. 58 (9): 894–901. doi:10.1016/j.jclinepi.2005.01.006. ISSN 0895-4356. PMID 16085192. <a href="https://www.jclinepi.com/article/S0895-4356(05)00082-X/abstract" target="_blank">https://www.jclinepi.com/article/S0895-4356(05)00082-X/abstract</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li>
</ol>

Funnel plot open-in-new

Funnel plot