Advanced searchCitation search
Skip main navigation
Close Drawer MenuOpen Drawer Menu
Research and Reporting Methods18 February 2014

Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change

    Author, Article, and Disclosure Information
    https://doi.org/10.7326/M13-2886

    This article has an additional interactive example appended as a Supplement. Please visit the Supplement tab on this page to access the presentation.

    A primary goal of meta-analysis is to improve the estimation of treatment effects by pooling results of similar studies. This article explains how the most widely used method for pooling heterogeneous studies—the DerSimonian–Laird (DL) estimator—can produce biased estimates with falsely high precision. A classic example is presented to show that use of the DL estimator can lead to erroneous conclusions. Particular problems with the DL estimator are discussed, and several alternative methods for summarizing heterogeneous evidence are presented. The authors support replacing universal use of the DL estimator with analyses based on a critical synthesis that recognizes the uncertainty in the evidence, focuses on describing and explaining the probable sources of variation in the evidence, and uses random-effects estimates that provide more accurate confidence limits than the DL estimator.

    References

    • 1. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177-88. [PMID: 3802833] CrossrefMedlineGoogle Scholar
    • 2. Weseley AC, Douglas GW. Continuous use of chlorothiazide for prevention of toxemia of pregnancy. Obstet Gynecol. 1962;19:355-8. [PMID: 14006267] MedlineGoogle Scholar
    • 3. Flowers CE, Grizzle JE, Easterling WE, Bonner OB. Chlorothiazide as a prophylaxis against toxemia of pregnancy. A double-blind study. Am J Obstet Gynecol. 1962;84:919-29. [PMID: 13893680] CrossrefMedlineGoogle Scholar
    • 4. Menzies DN. Controlled trial of chlorothiazide in treatment of early pre-eclampsia. Br Med J. 1964;1:739-42. [PMID: 14102017] CrossrefMedlineGoogle Scholar
    • 5. Fallis NE, Plauche WC, Mosey LM, Langford HG. Thiazide versus placebo in prophylaxis of toxemia of pregnancy in primigravid patients. Am J Obstet Gynecol. 1964;88:502-4. [PMID: 14123429] CrossrefMedlineGoogle Scholar
    • 6. Cuadros A, Tatum HJ. The prophylactic and therapeutic use of bendroflumethiazide in pregnancy. Am J Obstet Gynecol. 1964;89:891-7. [PMID: 14207556] CrossrefMedlineGoogle Scholar
    • 7. Landesman R, Aguero O, Wilson K, LaRussa R, Campbell W, Penaloza O. The prophylactic use of chlorthalidone, a sulfonamide diuretic, in pregnancy. Br J Obstet Gynaecol. 1965;72:1004-10. CrossrefGoogle Scholar
    • 8. Kraus GW, Marchese JR, Yen SS. Prophylactic use of hydrochlorothiazide in pregnancy. JAMA. 1966;198:1150-4. [PMID: 5332983] CrossrefMedlineGoogle Scholar
    • 9. Tervilä L, Vartiainen E. The effects and side effects of diuretics in the prophylaxis of toxaemia of pregnancy. Acta Obstet Gynecol Scand. 1971;50:351-6. [PMID: 4945572] CrossrefMedlineGoogle Scholar
    • 10. Campbell DM, MacGillivray I. The effect of a low calorie diet or a thiazide diuretic on the incidence of pre-eclampsia and on birth weight. Br J Obstet Gynaecol. 1975;82:572-7. [PMID: 1096930] CrossrefMedlineGoogle Scholar
    • 11. Collins R, Yusuf S, Peto R. Overview of randomised trials of diuretics in pregnancy. Br Med J (Clin Res Ed). 1985;290:17-23. [PMID: 3917318] CrossrefMedlineGoogle Scholar
    • 12. Biggerstaff BJ, Tweedie RL. Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. Stat Med. 1997;16:753-68. [PMID: 9131763] CrossrefMedlineGoogle Scholar
    • 13. Hardy RJ, Thompson SG. A likelihood approach to meta-analysis with random effects. Stat Med. 1996;15:619-29. [PMID: 8731004] CrossrefMedlineGoogle Scholar
    • 14. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21:1539-58. [PMID: 12111919] CrossrefMedlineGoogle Scholar
    • 15. Review Manger (RevMan) [computer program]. Version 5.2. Copenhagen, Denmark: Nordic Cochrane Center, Cochrane Collaboration; 2012. Google Scholar
    • 16. DerSimonian R, Kacker R. Random-effects model for meta-analysis of clinical trials: an update. Contemp Clin Trials. 2007;28:105-14. [PMID: 16807131] CrossrefMedlineGoogle Scholar
    • 17. Knapp G, Hartung J. Improved tests for a random effects meta-regression with a single covariate. Stat Med. 2003;22:2693-710. [PMID: 12939780] CrossrefMedlineGoogle Scholar
    • 18. Normand SL. Meta-analysis: formulating, evaluating, combining, and reporting. Stat Med. 1999;18:321-59. [PMID: 10070677] CrossrefMedlineGoogle Scholar
    • 19. Whitehead A. Meta-analysis of Controlled Clinical Trials. New York: J Wiley; 2002. Google Scholar
    • 20. Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A Stat Soc. 2009;172:137-59. [PMID: 19381330] CrossrefMedlineGoogle Scholar
    • 21. Sterne JAC. Meta-analysis in Stata: An Updated Collection from the Stata Journal. 1st ed. College Station, Texas: Stata Pr; 2009. Google Scholar
    • 22. Viechtbauer W. Conducting meta-analysis in R with the metafor package. J Stat Softw. 2010;36:1-48. CrossrefGoogle Scholar
    • 23. Brockwell SE, Gordon IR. A comparison of statistical methods for meta-analysis. Stat Med. 2001;20:825-40. [PMID: 11252006] CrossrefMedlineGoogle Scholar
    • 24. Kontopantelis E, Reeves D. metaan: Random-effects meta-analysis. Stata J. 2010;10:395-407. CrossrefGoogle Scholar
    • 25. Guolo A, Varin C. The R package metaLik for likelihood inference in meta-analysis. J Stat Softw. 2012;50:1-14. CrossrefMedlineGoogle Scholar
    • 26. Guolo A. Higher-order likelihood inference in meta-analysis and meta-regression. Stat Med. 2012;31:313-27. [PMID: 22173666] CrossrefMedlineGoogle Scholar
    • 27. Warn DE, Thompson SG, Spiegelhalter DJ. Bayesian random effects meta-analysis of trials with binary outcomes: methods for the absolute risk difference and relative risk scales. Stat Med. 2002;21:1601-23. [PMID: 12111922] CrossrefMedlineGoogle Scholar
    • 28. Lambert PC, Sutton AJ, Burton PR, Abrams KR, Jones DR. How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS. Stat Med. 2005;24:2401-28. [PMID: 16015676] CrossrefMedlineGoogle Scholar
    • 29. Higgins JP, Whitehead A. Borrowing strength from external trials in a meta-analysis. Stat Med. 1996;15:2733-49. [PMID: 8981683] CrossrefMedlineGoogle Scholar
    Previous
    Next

    Submit a Comment

    Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
    *All comments submitted after October 1, 2021 and selected for publication will be published online only.
    Cancel