Meta-analysis

Robust Bayesian (publication-bias adjusted)

Robust Bayesian meta-analysis model-averages over the presence or absence of an effect, of heterogeneity and of publication bias, reporting inclusion Bayes factors and a bias-adjusted pooled effect.

What is Robust Bayesian (publication-bias adjusted)?

A conventional meta-analysis commits to a fixed- or random-effects model and, at most, tests for publication bias as an afterthought. The robust Bayesian version turns each of those choices into a question and averages across them. It builds an ensemble by crossing three components: an effect (μ ~ Normal(0, 1) vs. μ = 0), heterogeneity (τ ~ Inverse-Gamma vs. τ = 0), and publication bias (none, vs. selection weight-functions on the study p-values, vs. PET / PEESE small-study-effect regressions). Every model's marginal likelihood is computed by deterministic quadrature, and the models are combined by their posterior probabilities.

Each of the three questions gets an INCLUSION Bayes factor — the posterior odds that the component is present divided by the prior odds (½ vs. ½, so 1). BF_effect answers 'is there a non-zero pooled effect?', BF_heterogeneity answers 'do the true effects vary across studies?', and BF_pubbias answers 'is there evidence of publication bias / small-study effects?'.

The model-averaged pooled effect and its credible interval already account for uncertainty about heterogeneity and bias: when the data show funnel asymmetry, the selection and PET-PEESE models gain posterior weight and pull the estimate toward its bias-corrected value; when the data are symmetric, the no-bias models dominate and the estimate matches an ordinary Bayesian meta-analysis. Restricting the ensemble to the no-bias models reproduces a fixed-effect (τ = 0) or random-effects Bayesian meta-analysis exactly.

When should I use Robust Bayesian (publication-bias adjusted)?

  • Synthesizing per-study effect sizes (with standard errors) when publication bias or small-study effects are a concern.
  • When you want a single analysis that reports evidence for the effect while hedging over heterogeneity and bias, rather than running a pooled estimate and a separate funnel/Egger test.
  • Bodies of literature where selective reporting of significant results is plausible.

What data does it need?

Two numeric columns: effect size yᵢ and its standard error seᵢ (+ optional study labels), with adjustable effect and heterogeneity prior widths.

What does it report?

Three inclusion Bayes factors (effect / heterogeneity / publication bias) with posterior inclusion probabilities, a model-averaged pooled effect with 95% credible interval (plus the effect-present conditional estimate), and a per-model ensemble table (relative log marginal likelihood + posterior probability).

What does it assume?

  • Per-study effect sizes are on a common scale with correctly computed standard errors, and studies are independent.
  • The Normal effect prior, Inverse-Gamma heterogeneity prior, and the selection / PET-PEESE bias models are sensible defaults; rerun with a wider effect prior to check sensitivity.

How do I interpret the result?

BF_effect > 1 favours a non-zero pooled effect; BF_heterogeneity > 1 favours between-study variation; BF_pubbias > 1 favours the presence of publication bias. Read each on the Jeffreys scale (3 / 10 / 100).

A publication-bias BF well above 1 means the naive pooled estimate is likely inflated — the model-averaged estimate reported here already discounts it.

The ensemble table shows which combinations of assumptions the data support; a single dominant row means the averaging is effectively selecting one model.

See also

References

  • Maier, Bartoš & Wagenmakers (2023). Robust Bayesian meta-analysis: Addressing publication bias with model-averaging. Psychological Methods 28(1):107-122.
  • Bartoš, Maier, Wagenmakers, Doucouliagos & Stanley (2023). Robust Bayesian meta-analysis: Model-averaging across complementary publication-bias adjustment methods. Research Synthesis Methods 14(1):99-116.
  • Vevea & Hedges (1995). A general linear model for estimating effect size in the presence of publication bias. Psychometrika 60:419-435.
  • Stanley & Doucouliagos (2014). Meta-regression approximations to reduce publication selection bias. Research Synthesis Methods 5(1):60-78.
  • Kass & Raftery (1995). Bayes factors. JASA 90(430):773-795.