Stefano Garzon

Statistics · Oct 23, 2026

Meta-analysis: fixed effect, random effects, heterogeneity and publication bias

Concept

A meta-analysis combines the results of several studies into a single estimate. It differs from a narrative review because it attaches a p value to its conclusion. Friedman et al. note that this value can convey more precision than the evidence supports.

The validity of the pooled estimate depends on three factors: which studies enter, how they are weighted, and whether the studies are comparable.

Fixed effect and random effects

The fixed-effect model, such as the Mantel-Haenszel method, assumes all studies estimate one true effect. Differences between studies reflect only sampling error. Each study is weighted by the inverse of its variance, wi=1/viw_i = 1/v_i. Large studies receive more weight.

The random-effects model, such as DerSimonian and Laird’s, assumes the true effect varies across studies. The between-study variance, τ2\tau^2, enters the weight:

wi∗=1vi+τ2w_i^* = \frac{1}{v_i + \tau^2}

The pooled estimate is the weighted mean θ^=∑wiθ^i/∑wi\hat{\theta} = \sum w_i \hat{\theta}_i / \sum w_i.

When τ2\tau^2 is large, weights become more similar across studies. For this reason the random-effects model can give small studies disproportionate influence. Friedman et al. state a preference for the fixed-effect model and recommend running both to check whether the conclusion depends on the weighting. There is no consensus on which model is correct.

Bayesian analysis is a third valid, less used, form of synthesis. The forest plot is the standard graphical presentation: each study’s effect and confidence interval, with the pooled estimate.

Heterogeneity

Studies differ in population, intervention, outcome and logistics. Some inconsistency between results is expected. Cochran’s QQ statistic tests homogeneity. The power of this test depends on the number of studies. Higgins and Thompson proposed the statistic I2=(Q−df)/QI^2 = (Q - df)/Q, where dfdf is the number of studies minus 1. It describes the proportion of total variation in study estimates attributable to heterogeneity, and does not depend on the number of studies or the effect metric. The authors recommend reporting it in preference to the heterogeneity test.

Friedman et al. consider examination of clinical differences between studies more informative than the formal test. Study characteristics, such as population age, treatment duration or study size, can explain part of the heterogeneity.

Quantitative heterogeneity is a difference in magnitude with the effect in the same direction. Qualitative heterogeneity occurs when some studies show benefit and others harm. With qualitative heterogeneity, a pooled estimate is not advisable, especially when it reflects a time trend in dose or population.

Study selection and publication bias

The choice of included studies is one of the main threats to validity. The defensible approach considers all relevant studies conducted. Meta-analyses of the same question with different inclusion criteria can reach different conclusions. A prespecified protocol reduces investigator bias.

Publication bias is the tendency of neutral or negative trials to remain unpublished, or to be published late. A systematic search of a standard database can miss a relevant share of published trials. Simes compared published trials with trials registered in advance in a cancer registry. In advanced ovarian cancer, published trials showed a survival advantage for combination chemotherapy (median survival ratio 1.16; p = 0.02). Registered trials showed no significant difference (1.05; p = 0.25). The funnel plot is the usual tool to inspect asymmetry compatible with this bias. Egger et al. proposed a regression test for this asymmetry. In eight pairs of a meta-analysis and a later large trial, four were discordant, all with a larger effect in the meta-analysis; asymmetry was present in three of the four discordant pairs and in none of the concordant pairs. The test has little power when the meta-analysis includes few small trials.

Data quality also varies across trials. For this reason meta-analysis works best with simple, objective outcomes, such as all-cause mortality or major events.

Common errors when reading a paper

What to check in a meta-analysis

References