Skip main navigation
Close Drawer MenuOpen Drawer Menu
Research and Reporting Methods15 August 2017

Sensitivity Analysis in Observational Research: Introducing the E-Value

    Author, Article and Disclosure Information
    https://doi.org/10.7326/M16-2607
    Visual Abstract
    Full Text
    PDF
    Tools

    Sensitivity analysis is useful in assessing how robust an association is to potential unmeasured or uncontrolled confounding. This article introduces a new measure called the “E-value,” which is related to the evidence for causality in observational studies that are potentially subject to confounding. The E-value is defined as the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment–outcome association, conditional on the measured covariates. A large E-value implies that considerable unmeasured confounding would be needed to explain away an effect estimate. A small E-value implies little unmeasured confounding would be needed to explain away an effect estimate. The authors propose that in all observational studies intended to produce evidence for causality, the E-value be reported or some other sensitivity analysis be used. They suggest calculating the E-value for both the observed association estimate (after adjustments for measured confounders) and the limit of the confidence interval closest to the null. If this were to become standard practice, the ability of the scientific community to assess evidence from observational studies would improve considerably, and ultimately, science would be strengthened.

    References

    • 1. Wasserstein RL  and  Lazar NL . The ASA's statement on p-values: context, process, and purpose. Am Stat. 2016;70:129-33. CrossrefGoogle Scholar
    • 2. Altman DG ,  Machin D ,  Bryant TN , and  Gardner MJ . Statistics with Confidence. 2nd ed. London BMJ Books 2000. Google Scholar
    • 3. Rosner B . Fundamentals of Biostatistics. 8th ed. Boston Cengage Learning 2015. Google Scholar
    • 4. Pagano M  and  Gavreau K . Principles of Biostatistics. Belmont, CA Brooks/Cole 2000. Google Scholar
    • 5. Greenland S ,  Senn SJ ,  Rothman KJ ,  Carlin JB ,  Poole C ,  Goodman SN , et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31:337-50. [PMID: 27209009] doi:10.1007/s10654-016-0149-3 CrossrefMedlineGoogle Scholar
    • 6. Goodman S . A dirty dozen: twelve p-value misconceptions. Semin Hematol. 2008;45:135-40. [PMID: 18582619] doi:10.1053/j.seminhematol.2008.04.003 CrossrefMedlineGoogle Scholar
    • 7. Greenland S . Null misinterpretation in statistical testing and its impact on health risk assessment. Prev Med. 2011;53:225-8. [PMID: 21871481] doi:10.1016/j.ypmed.2011.08.010 CrossrefMedlineGoogle Scholar
    • 8. Greenland S  and  Poole C . Problems in common interpretations of statistics in scientific articles, expert reports, and testimony. Jurimetrics. 2011;51:11329. Google Scholar
    • 9. Sterne JA  and  DaveySmith G . Sifting the evidence—what's wrong with significance tests? BMJ. 2001;322:226-31. [PMID: 11159626] CrossrefMedlineGoogle Scholar
    • 10. Stang A ,  Poole C , and  Kuss O . The ongoing tyranny of statistical significance testing in biomedical research. Eur J Epidemiol. 2010;25:225-30. [PMID: 20339903] doi:10.1007/s10654-010-9440-x CrossrefMedlineGoogle Scholar
    • 11. Goodman SN . Toward evidence-based medical statistics. 1: The P value fallacy. Ann Intern Med. 1999;130:995-1004 LinkGoogle Scholar
    • 12. Hill AB . The environment and disease: association or causation? Proc R Soc Med. 1965;58:295-300. [PMID: 14283879] CrossrefMedlineGoogle Scholar
    • 13. Greenland S . Randomization, statistics, and causal inference. Epidemiology. 1990;1:421-9. [PMID: 2090279] CrossrefMedlineGoogle Scholar
    • 14. Imbens GW  and  Rubin DB . Sensitivity analysis and bounds. In: Causal Inference for Statistics, Social, and Biomedical Sciences. New York Cambridge Univ Pr 2015 496-512. Google Scholar
    • 15. Hernan MA, Robins JR. Confounding. In: Causal Inference. 11 September 2016. Accessed at https://cdn1.sph.harvard.edu/wp-content/uploads/sites/1268/2016/09/hernanrobins_v1.10.31.pdf on 22 May 2017. Google Scholar
    • 16. Rosenbaum PR . Sensitivity to hidden bias. In: Observational Studies. 2nd ed. New York Springer 2002 105-70. Google Scholar
    • 17. Rosenbaum PR . Design sensitivity. In: Design of Observational Studies. New York Springer 2010 269-71. Google Scholar
    • 18. Rosenbaum PR . Design sensitivity and efficiency in observational studies. J Am Stat Assoc. 2010;105:692-702. CrossrefGoogle Scholar
    • 19. Greenland S . Multiple-bias modeling for analysis of observational data. J R Stat Soc Ser A. 2005;168:267-308. CrossrefGoogle Scholar
    • 20. Lash TL ,  Fox MP , and  Fink AK . Applying Quantitative Bias Analysis to Epidemiologic Data. New York Springer 2009. Google Scholar
    • 21. Greenland S  and  Lash TL . Bias analysis.. In: Rothman KJ, Greenland S, Lash TL, eds. Modern Epidemiology. Philadelphia Lippincott Williams & Wilkins 2008 345-80. Google Scholar
    • 22. Cornfield J ,  Haenszel W ,  Hammond EC ,  Lilienfeld AM ,  Shimkin MB , and  Wynder EL . Smoking and lung cancer: recent evidence and a discussion of some questions. J Natl Cancer Inst. 1959;22:173-203. [PMID: 13621204] MedlineGoogle Scholar
    • 23. Victora CG ,  Smith PG ,  Vaughan JP ,  Nobre LC ,  Lombardi C ,  Teixeira AM , et al. Evidence for protection by breast-feeding against infant deaths from infectious diseases in Brazil. Lancet. 1987;2:319-22. [PMID: 2886775] CrossrefMedlineGoogle Scholar
    • 24. Bross ID . Spurious effects from an extraneous variable. J Chronic Dis. 1966;19:637-47. [PMID: 5966011] CrossrefMedlineGoogle Scholar
    • 25. Schlesselman JJ . Assessing effects of confounding variables. Am J Epidemiol. 1978;108:3-8. [PMID: 685974] MedlineGoogle Scholar
    • 26. Rosenbaum PR  and  Rubin DB . Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. J R Stat Soc Series B Stat Methodol. 1983;45:212-8. Google Scholar
    • 27. Lin DY ,  Psaty BM , and  Kronmal RA . Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics. 1998;54:948-63. [PMID: 9750244] CrossrefMedlineGoogle Scholar
    • 28. Imbens GW . Sensitivity to exogeneity assumptions in program evaluation. Am Econ Rev. 2003;93:126-32. CrossrefGoogle Scholar
    • 29. Vanderweele TJ  and  Arah OA . Bias formulas for sensitivity analysis of unmeasured confounding for general outcomes, treatments, and confounders. Epidemiology. 2011;22:42-52. [PMID: 21052008] doi:10.1097/EDE.0b013e3181f74493 CrossrefMedlineGoogle Scholar
    • 30. Bross ID . Pertinency of an extraneous variable. J Chronic Dis. 1967;20:487-95. [PMID: 6028268] CrossrefMedlineGoogle Scholar
    • 31. Lee WC . Bounding the bias of unmeasured factors with confounding and effect modifying potentials. Stat Med. 2011;30:1007-17. CrossrefMedlineGoogle Scholar
    • 32. Robins J M ,  Scharfstein D , and  Rotnitzky A . Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models.. In: Halloran E, Berry D, eds. Statistical Models for Epidemiology, the Environment, and Clinical Trials. New York Springer-Verlag 2000 1-94. Google Scholar
    • 33. McCandless LC ,  Gustafson P , and  Levy A . Bayesian sensitivity analysis for unmeasured confounding in observational studies. Stat Med. 2007;26:2331-47. [PMID: 16998821] CrossrefMedlineGoogle Scholar
    • 34. Brumback BA ,  Hernán MA ,  Haneuse SJ , and  Robins JM . Sensitivity analyses for unmeasured confounding assuming a marginal structural model for repeated measures. Stat Med. 2004;23:749-67. [PMID: 14981673] CrossrefMedlineGoogle Scholar
    • 35. VanderWeele TJ . Sensitivity analysis for mediation. In: Explanation in Causal Inference: Methods for Mediation and Interaction. New York Oxford Univ Pr 2015 66-97. Google Scholar
    • 36. VanderWeele TJ . Sensitivity analysis for contagion effects in social networks. Sociol Methods Res. 2011;40:240-55. [PMID: 25580037] CrossrefMedlineGoogle Scholar
    • 37. Ding P  and  VanderWeele TJ . Sensitivity analysis without assumptions. Epidemiology. 2016;27:368-77. [PMID: 26841057] doi:10.1097/EDE.0000000000000457 CrossrefMedlineGoogle Scholar
    • 38. Gilbert PB ,  Bosch RJ , and  Hudgens MG . Sensitivity analysis for the assessment of causal vaccine effects on viral load in HIV vaccine trials. Biometrics. 2003;59:531-41. [PMID: 14601754] CrossrefMedlineGoogle Scholar
    • 39. Chiba Y  and  VanderWeele TJ . A simple method for principal strata effects when the outcome has been truncated due to death. Am J Epidemiol. 2011;173:745-51. [PMID: 21354986] doi:10.1093/aje/kwq418 CrossrefMedlineGoogle Scholar
    • 40. Huang TH  and  Lee WC . Bounding formulas for selection bias. Am J Epidemiol. 2015;182:868-72. [PMID: 26519426] doi:10.1093/aje/kwv130 CrossrefMedlineGoogle Scholar
    • 41. Rosenbaum PR . Discussing hidden bias in observational studies. Ann Intern Med. 1991;115:901-5 LinkGoogle Scholar
    • 42. Ip S ,  Chung M ,  Raman G ,  Chew P ,  Magula N ,  DeVine D , et al. Breastfeeding and Maternal and Infant Health Outcomes in Developed Countries. Evidence Report/Technology Assessment no. 153. (Prepared by Tufts-New England Medical Center Evidence-based Practice Center under contract no. 290-02-0022.) AHRQ publication no. 07-E007. Rockville Agency for Healthcare Research and Quality April 2007. Google Scholar
    • 43. Moorman PG ,  Calingaert B ,  Palmieri RT ,  Iversen ES ,  Bentley RC ,  Halabi S , et al. Hormonal risk factors for ovarian cancer in premenopausal and postmenopausal women. Am J Epidemiol. 2008;167:1059-69. [PMID: 18303003] doi:10.1093/aje/kwn006 CrossrefMedlineGoogle Scholar
    • 44. Taubes G . Epidemiology faces its limits. Science. 1995;269:164-9. [PMID: 7618077] CrossrefMedlineGoogle Scholar
    • 45. VanderWeele TJ. On a square-root transformation of the odds ratio for a common outcome. Epidemiology. 2017. [Forthcoming]. Google Scholar
    • 46. Borenstein M ,  Hedges LV ,  Higgins JPT , and  Rothstein HR . Converting among effect sizes. In: Introduction to Meta-Analysis. Hoboken, NJ Wiley 2009 45-51. Google Scholar
    • 47. Hasselblad V  and  Hedges LV . Meta-analysis of screening and diagnostic tests. Psychol Bull. 1995;117:167-78. [PMID: 7870860] CrossrefMedlineGoogle Scholar
    Previous
    Next

    Submit a Comment

    Contributors must reveal any conflict of interest. Comments are moderated.
    Cancel