What a Funnel Plot Shows, and What It Cannot

A funnel plot is a scatter graph that a meta-analysis uses to ask one quiet question: are some of the studies that should exist missing from the picture? It plots each study by its effect on one axis and its precision on the other, then invites you to look for a gap where small, unflattering results should sit. A symmetric plot is reassuring and an asymmetric one is a prompt to investigate, yet the plot alone never proves that a study was suppressed. This is a method article, not medical advice; for decisions about your own care, talk with a clinician who knows your history.

I read these plots the way a careful reviewer reads anything labeled reassuring, slowly and assuming the easy reading is wrong. When I co-authored a systematic review and meta-analysis in Diabetes Care, the work that taught me the most was the question of what we might never have found at all.

The shape the plot is built to reveal

Imagine the precise studies near the top of the figure and the imprecise ones spread along the bottom. Large studies estimate the effect tightly, so they cluster in a narrow band high on the plot. Small studies are noisier, so they scatter far to the left and right lower down. With no distortion, that pattern forms a roughly symmetric inverted funnel, wide at the base and narrow at the peak.

The funnel is a sampling expectation, not a law. If every study ever run had an equal chance of reaching print, the small ones would miss the true effect in both directions by chance alone, landing on either side in roughly equal numbers. The symmetry you hope to see is simply that even-handed scatter made visible.

Why a missing corner matters

Publication bias is the tendency for results that look interesting to reach a journal while null or unwelcome ones stay in a drawer. A small study that found a strong effect gets written up and published. A small study that found nothing often does not, because neither the authors nor the editors felt the pull.

That selective drift leaves a recognizable hole. If the small null studies were quietly filtered out, one lower corner of the funnel sits emptier than its mirror image, and the cloud tilts toward the side that favors an effect. Reading the gap is the whole point. You are looking for the studies that are not there by noticing the shape of the space they would have occupied.

What asymmetry can suggest

An asymmetric funnel raises the possibility that the pooled estimate is built on a filtered sample. If the small studies on the unfavorable side are missing, the average of the studies that remain drifts away from the truth, and a confident-looking summary may be confident about a biased pool. That is the alarm the plot is designed to ring.

Asymmetry also flags what reviewers call small study effects, the broader pattern of smaller studies reporting systematically larger effects than larger ones. Publication bias is the cause readers reach for first, but it is not the only one, which is why the plot is a starting point rather than a conclusion.

What asymmetry cannot tell you

A lopsided funnel does not prove that anyone hid anything. Several honest mechanisms produce the same picture, and the plot cannot distinguish among them. Small studies often enroll sicker or more selected groups, or apply a treatment more intensively, so they may genuinely find larger effects for real clinical reasons.

Methodological quality tracks with size as well. Smaller studies are, on average, more prone to design weaknesses that inflate effects, so their results can reflect bias inside each study rather than bias in which studies survived. The funnel sees only effect and precision; it stays blind to the reason behind the shape.

The plot is also unreliable when there are few studies to plot. With a handful of points, the eye invents funnels and gaps that are no more than noise, and formal tests for asymmetry lose their footing too. A plot of five studies tells you almost nothing about symmetry, however suggestive it looks.

Heterogeneity muddies it further. When the true effect honestly differs across populations or designs, the studies spread for legitimate reasons, and that real variation can read as asymmetry that has nothing to do with missing work.

The tests that travel with the plot

Because eyeballing a scatter is subjective, reviewers add statistical tests that put a number on the tilt, the best known of which asks whether a study's effect is associated with its precision. A small probability value suggests the funnel is more lopsided than chance would explain, though it does not name the cause, so it inherits every limitation above.

Some analyses go further and estimate what the pooled result might have looked like had the suspected missing studies been present. I treat those adjustments as a sensitivity check, not a recovery of the truth. If the corrected estimate and the original tell the same story, the finding is robust to the worry. If they diverge, the honest report is that the evidence is shakier than the headline.

How I read one as a reviewer

I start by asking whether there are enough studies for the plot to mean anything, and if there are not, I disregard it rather than over-read it. Then I look for a tilt, and if I see one, I resist the reflex to call it bias and ask what else could produce it: differences in the populations, the quality of the small studies, the spread of true effects across settings.

The discipline is to hold the plot as one line of evidence among several. A funnel that leans should send you back to the methods, the registration, and the breadth of the search, never straight to a verdict. What persuades me is convergence: a tilted plot, a search that admitted only published English-language trials, and outcomes that drifted after the data arrived tell a story the funnel alone cannot.

A funnel plot earns its place by asking the one question arithmetic cannot answer, which is whether the raw material was complete before the pooling began. It is a smoke detector, not a fire investigation. A review that draws the plot, reads it carefully, and then keeps investigating has used it well. A review that draws a symmetric funnel and declares itself unbiased has mistaken the absence of an alarm for proof there was never a fire.