Acknowledgment: The authors thank Thomasin Adams-Webber (Hospital for Sick Children) for her help in designing the search strategy.
Disclosures: Dr. El Dib received a São Paulo Research Foundation (FAPESP) (2018/11205-6) scholarship and funding from the National Council for Scientific and Technological Development (CNPq) (CNPq 310953/2015-4) and the Faculty of Medicine, Dalhousie University. Dr. Johnston received a grant from Texas A&M AgriLife Research to fund investigator-driven research related to saturated and polyunsaturated fats within the 36-month reporting period required by the International Committee of Medical Journal Editors, as well as funding received from the International Life Science Institute (North America) that ended before the 36-month reporting period. Authors not named here have disclosed no conflicts of interest. Disclosures can also be viewed at
www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M19-0655.
Editors' Disclosures: Christine Laine, MD, MPH, Editor in Chief, reports that her spouse has stock options/holdings with Targeted Diagnostics and Therapeutics. Darren B. Taichman, MD, PhD, Executive Editor, reports that he has no financial relationships or interests to disclose. Cynthia D. Mulrow, MD, MSc, Senior Deputy Editor, reports that she has no relationships or interests to disclose. Jaya K. Rao, MD, MHS, Deputy Editor, reports that she has stock holdings/options in Eli Lilly and Pfizer. Catharine B. Stack, PhD, MS, Deputy Editor, Statistics, reports that she has stock holdings in Pfizer, Johnson & Johnson, and Colgate-Palmolive. Christina C. Wee, MD, MPH, Deputy Editor, reports employment with Beth Israel Deaconess Medical Center. Sankey V. Williams, MD, Deputy Editor, reports that he has no financial relationships or interests to disclose. Yu-Xiao Yang, MD, MSCE, Deputy Editor, reports that he has no financial relationships or interest to disclose.
>Reproducible Research Statement: Study protocol: Registered with PROSPERO (CRD42017074074).
Statistical code and data set: Available from Ms. Zeraatkar (e-mail,
[email protected]). For sample code, see
Supplement 2.
Corresponding Author: Bradley C. Johnston, PhD, Department of Community Health and Epidemiology, Faculty of Medicine, Dalhousie University, Room 404, 5790 University Avenue, Halifax, Nova Scotia B3J 0E4, Canada; e-mail,
[email protected].
Current Author Addresses: Ms. Zeraatkar, Drs. Guyatt and Hanna, and Ms. Bartoszko: Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada.
Dr. Han: Department of Preventive Medicine, College of Medicine, Chosun University, 309 Philmun-daero, Dong-gu, Gwangju 61452, Korea.
Dr. Vernooij: Department of Research, Netherlands Comprehensive Cancer Organisation, Godebaldkwartier 419, Utrecht 3511 DT, the Netherlands.
Dr. El Dib: Institute of Science and Technology, São José dos Campos, Avenida Engenheiro Francisco José Longo, 777, Jardim São Dimas, São José dos Campos, 12245-000, Spain.
Dr. Cheung: 114 Loganberry Crescent, Toronto, Ontario M2H 3H1, Canada.
Mr. Milio: 592 Regal Place, Waterloo, Ontario N2V 2G3, Canada.
Mr. Zworth: 28 York Downs Drive, Toronto, Ontario M3H 1J1, Canada.
Ms. Valli and Drs. Rabassa and Alonso-Coello: Iberoamerican Cochrane Centre, (IIB Sant Pau-CIBERESP), Carrer de Sant Antoni Maria Claret, 167, Barcelona, 08025, Spain.
Mr. Lee: 30 White Lodge Crescent, Richmond Hill, Ontario L4C 9A1, Canada.
Drs. Zajac, Prokop-Dorner, and Bala: Jagiellonian University Medical College, Department of Hygiene and Dietetics, Kopernika 7 Street, 31-034 Krakow, Poland.
Mr. Lo: 556 Amarone Court, Mississauga, Ontario L5W 0A7, Canada.
Dr. Johnston: Department of Community Health and Epidemiology, Faculty of Medicine, Dalhousie University, Room 404, 5790 University Avenue, Halifax, Nova Scotia B3J 0E4, Canada.
Author Contributions: Conception and design: D. Zeraatkar, G.H. Guyatt, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Analysis and interpretation of the data: D. Zeraatkar, M.A. Han, R.W.M. Vernooij, K. Milio, M. Rabassa, A. Prokop-Dorner, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Drafting of the article: D. Zeraatkar, M.A. Han, S.E. Hanna, B.C. Johnston.
Critical revision for important intellectual content: D. Zeraatkar, G.H. Guyatt, R.W.M. Vernooij, M. Rabassa, Y. Lee, A. Prokop-Dorner, C. Lo, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Final approval of the article: D. Zeraatkar, M.A. Han, G.H. Guyatt, R.W.M. Vernooij, R. El Dib, K. Cheung, K. Milio, M. Zworth, J.J. Bartoszko, C. Valli, M. Rabassa, Y. Lee, J. Zajac, A. Prokop-Dorner, C. Lo, M.M. Bala, P. Alonso-Coello, S.E. Hanna, B.C. Johnston.
Statistical expertise: D. Zeraatkar, S.E. Hanna, B.C. Johnston.
Administrative, technical, or logistic support: D. Zeraatkar, R. El Dib, Y. Lee, S.E. Hanna, B.C. Johnston.
Collection and assembly of data: D. Zeraatkar, M.A. Han, R.W.M. Vernooij, R. El Dib, K. Cheung, K. Milio, M. Zworth, J.J. Bartoszko, C. Valli, M. Rabassa, Y. Lee, J. Zajac, C. Lo, B.C. Johnston.
This article was published at
Annals.org on 1 October 2019.
The application of GRADE to nutritional epidemiology is appropriate
We would like to begin by correcting an error in Dr. Giovannuci and Rimm’s letter. They state: “it would be questionable to move even smoking and lung cancer risk up from ‘low’ evidence”. GRADE provides for high quality evidence from observational studies through two major considerations: convincing evidence of large effects and dose-response gradients. Unlike the association between red and processed meat and adverse health outcomes for which there is no convincing evidence of large relative risks (<0.5 or >2.0) (1), the magnitude of association between smoking and lung cancer is much larger, with relative risks ranging between 5 and 25 (2, 3)*. Moreover, there is also a credible dose-response gradient. Because smoking and lung cancer fulfills both criteria, the application of GRADE would infer high quality evidence.
We will now proceed to the major epistemological concern. The letter authors make the case that the epistemological rules by which we distinguish between what is true from what is speculative or untrue should differ depending on the type of evidence that is possible to bring to bear on an issue.
Consider the following situation. Two bodies of evidence, each a series of equally well-done observational studies. One body of evidence addresses a drug for which randomized trials are possible and another a nutritional intervention for which randomized trials are not feasible. According to the authors’ logic, the first would be considered low quality evidence and the second identical body of evidence would be considered high quality evidence.
This strikes us as fundamentally illogical. Rules for determining the quality of evidence should not depend on the feasibility of particular research designs and should be consistent across health fields.
If one accepts that standards of how we know what we know should be uniform across health fields, the next issue is what standards of evidence we should adopt.
The GRADE system, based on rigorous methodology described in a series of over 25 peer-reviewed articles, has been adopted by more than 110 international organizations (4, 5). These include Cochrane, and the World Health Organization, each of which regularly apply GRADE to systematic reviews of observational studies. In addressing the challenges of applying GRADE to observational studies, researchers from the environmental field have worked with GRADE on the application of the methods to environmental topics (6).
Until someone develops an alternative standard that surpasses the rigor of the GRADE system, and is more widely endorsed by the scientific community, GRADE should remain the current best approach to achieving consistency in epistemological inquiry across health fields.
*This sentence was corrected on March 16, 2020. The original sentence read: "Unlike the association between red and processed meat and adverse health outcomes for which relative risks range from 1.08 to 1.28 (1), the magnitude of association between smoking and lung cancer is much larger, with relative risks ranging between 5 and 25 (2,3)."
References
1) Johnston BC, Zeraatkar D, Han MA, Vernooij RW, Valli C, El Dib R, Marshall C, Stover PJ, Fairweather-Taitt S, Wójcik G, Bhatia F. Unprocessed red meat and processed meat consumption: dietary guideline recommendations from the nutritional recommendations (NutriRECS) consortium. Annals of Internal Medicine. 2019 Oct 1.
2) Ordóñez-Mena JM, Schöttker B, Mons U, Jenab M, Freisling H, Bueno-de-Mesquita B, O’Doherty MG, Scott A, Kee F, Stricker BH, Hofman A. Quantification of the smoking-associated cancer risk with rate advancement periods: meta-analysis of individual participant data from cohorts of the CHANCES consortium. BMC medicine. 2016 Dec;14(1):62.
3) Pesch B, Kendzia B, Gustavsson P, Jöckel KH, Johnen G, Pohlabeln H, Olsson A, Ahrens W, Gross IM, Brüske I, Wichmann HE. Cigarette smoking and lung cancer—relative risk estimates for the major histological types from a pooled analysis of case–control studies. International journal of cancer. 2012 Sep 1;131(5):1210-9.
4) Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008 Apr 24;336(7650):924-6.
5) Guyatt GH, Oxman AD, Schünemann HJ, Tugwell P, Knottnerus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. Journal of clinical epidemiology. 2011 Apr 1;64(4):380-2.
6) Morgan RL, Beverly B, Ghersi D, Schünemann HJ, Rooney AA, Whaley P, Zhu YG, Thayer KA. GRADE guidelines for environmental and occupational health: A new series of articles in Environment International. Environment international. 2019 Jul;128:11.
Evaluation Criteria Doesn't Make the GRADE
References
1. Zeraatkar, D., et al., Red and processed meat consumption and risk for all-cause mortality and cardiometabolic outcomes: a systematic review and meta-analysis of cohort studies. Ann Intern Med, 2019.
2. Siemieniuk, R. and G. Guyatt. What is GRADE? BMJ Best Practice [cited 2019; Available from: https://bestpractice.bmj.com/info/us/toolkit/learn-ebm/what-is-grade/.
3. Katz, D.L., et al., Hierarchies of evidence applied to lifestyle medicine (HEALM): introduction of a strength-of-evidence approach based on a methodological systematic review. BMC Med Res Methodol, 2019. 19(1): p. 178.
4. Dietary Guidelines Advisory Committee, Scientific Report of the 2015 Dietary Guidelines Advisory Committee, US Department of Agriculture and US Department of Health & Human Services, Editors. 2015.
5. Hamblin, J., The actual reason meat is not healthy: nutrition studies leave out a crucial factor, in The Atlantic. 2019: Health.
We Are All Now Dumber.
If the goal was to publish a study, which would receive media attention - the authors and journal have done just that. I wish I could reassure the general pubic and my medical colleagues that reading through the remainder of the paper produced a level of confidence in the conclusion – it does not!
Both CAD and CA are associated with a series of inflammatory processes [4], which ultimately result in increased morbidity and mortality, missed by this retrospective review of what others have charted. This paper does a disservice to the journal, as well as medicine, the media and the general public. At a time when it is obvious that refined carbohydrates – including but not limited to sugar – as well as processed foods including meats and other foods, are a major contributor to CAD, CA, type 2 diabetes mellitus, high blood pressure and a host of other chronic inflammatory diseases, we should be conducting prospective research [2] into the causes and treatments of these diseases and not looking for retrospective support of bias.
I am sadly reminded of these words from Billy Madison:
“Mr. Madison, what you just said is one of the most insanely idiotic things I have ever heard. At no point in your rambling incoherent response were you even close to anything that could be considered a rational thought. Everyone in this room is now dumber for having listened to it. I award you no points and may God have mercy on your soul.”
Based upon the media coverage of this paper, we are all now dumber for having read and listened to what the authors published in the Annals of Internal Medicine.
Acknowledgment: FMTVDM [1] is issued to author.
References:
1. The Fleming Method for Tissue and Vascular Differentiation and Metabolism (FMTVDM) using same state single or sequential quantification comparisons. Patent Number 9566037. Issued 02/14/2017.
2. Fleming RM, Fleming MR, Chaudhuri TK. Are we prescribing the right diets and drugs for CAD, T2D, Cancer and Obesity? Int J Nuclear Med Radioactive Subs 2019;2(1):000115.
3. Zeraatkar D, Han A, Guyatt GH, et al. Red and Processed Meat Consumption and Risk for All-Cause Mortality and Cardiometabolic Outcomes. A Systematic Review and Meta-analysis of Cohort Studies. Ann Intern Med 2019. DOI:10.7326/M19-0655.
4. Fleming RM. Chapter 64. The Pathogenesis of Vascular Disease. Textbook of Angiology. John C. Chang Editor, Springer-Verlag New York, NY. 1999, pp. 787-798.
Disclosures: FMTVDM patent was issued to primary author.