Plants can't run and plants can't hide. Is it logical that they will selectively develop constituents which will be distasteful to their predators? These would presumably be chemicals of one kind or another. If they are harmful to their predators, could they be harmful to humans. This has been studied and proven to be true. Many of the chemicals are carcinogens. Nutritional value aside, it would appear that "organic" products probably have a greater potential for harm than those which are nurtured with a more flavorful, appealing product in mind. With no gain either way, the non-organic product wins my vote.

David E. Mouton, MD, Kelsey-Seybold Clinic, Houston, Texas

To the editor:

As an internist who relies on the Annals of Internal Medicine to publish articles that are free from bias and for which authors’ potential conflicts of interest are clearly stated, I was dismayed that the article by Smith-Spangler et al on organic food (1) did not indicate that some of the authors are affiliated with Stanford’s Freeman Spogli Institute which receives funding from agribusiness and agricultural chemical companies, such as Cargill and Monsanto. These affiliations may explain why the title, abstract and authors’ conclusions emphasize that “the evidence does not suggest marked health benefits from consuming organic versus conventional foods (1).” In actuality, the data demonstrate that organic foods have significantly fewer pesticide and toxic chemical residues and antibiotic resistant bacteria than conventionally grown food. Why were these important, findings not highlighted? The President’s Cancer Panel Report released in April, 2010 (2), states that exposure to EPA approved agricultural chemicals has been linked to cancers in most organ systems. The report advises individuals to take steps to reduce their cancer risk by minimizing their exposures to environmental toxins. The recommendations section states that “Individuals and families have many opportunities to reduce or eliminate chemical exposures. For example ... [e]xposure to pesticides can be decreased by choosing, to the extent possible, food grown without pesticides or chemical fertilizers (2).” Given the Panel’s recommendation, why did the authors not highlight this important evidence of benefit of organic food, rather than bury it in the body of the article where journalists and other readers are less likely to notice it?

Sari Lisa Davison, MD  Seattle, WA

References

1. Smith-Spangler, C., et al. (2012). Are organic foods safer or healthier than conventional alternatives? A systematic review. Ann Intern Med., 157(5),348-366.

2. Reuben, S. H., National Cancer Institute. (2010). Reducing environmental cancer risk: What we can do now: President's Cancer Panel 2008-2009 annual report. Bethesda, MD: President's Cancer Panel, p 111-112.

To the Editor:

Smith-Spangler et al. (1) state in their methods that they “evaluated the extent to which the organic–conventional comparison pairs were of the same cultivar or breed, grown on neighboring farms, and harvested during the same season” (p. 349), but in their results they admitted that they included a significant percentage of pairs in their meta-analysis that compared different cultivars and that did not come from neighboring fields (“Among produce studies, 59% … and 65% … compared food pairs from neighboring farms or the same cultivar, respectively.” p. 353). Thus, 41% and 35% of the pairs were not grown on neighboring farms or did not compare identical cultivars, respectively. It is well known that genetics and environmental factors influence plant metabolism, and hence the accumulation of phytonutrients. The importance of this was clearly pointed out by Lester and Saftner (2), yet the authors did not rigorously screen for only valid pairs that compared identical cultivars, growing in comparable soils and micro-environments. If the authors had carefully screened for cultivar and environmental factors, they may have found significant P values and homogeneity (i.e. non-significant heterogeneity statistic) for several nutrients reported in Table 1. The authors also failed to include a 2010 study (3) in their analysis that compared organically and conventionally grown strawberries in California in which cultivar and environmental factors were meticulously controlled. The authors of the PLoS ONE study found increased concentrations of vitamin C and total phenolic compounds, as well as higher anti-oxidant capacity in organic strawberries. (For the sake of full disclosure, I am a co-author of this PLoS ONE study.)The inclusion of invalid organic-conventional comparisons (i.e. those of different cultivars and/or growing in different soils and micro-environments) in this meta-analysis and the omission of the aforementioned study, which one of the co-authors admitted was “erroneously” omitted [4]), calls into question the results of this meta-analysis and the authors’ conclusions regarding nutritional differences between organic and conventional produce. Interested readers should contrast the results of this study with those of another meta-analysis (5), in which significantly higher concentrations of the same nutrients were reported for organically grown fruits and vegetables.

Preston K. Andrews, Ph.D.Associate Professor of Horticulture, Washington State University

References

1. Smith-Spangler C, Brandeau ML, Hunter GE, Bavinger JC, Pearson M, Eschbach PJ, et al. Are organic foods safer or healthier than conventional alternatives? A systematic review. Annals of Internal Medicine 2012;157:348-366.

2. Lester GE, Saftner RA. Organically versus conventionally grown produce: Common production inputs, nutritional quality, and nitrogen delivery between the two systems. Journal of Agriculture and Food Chemistry 2011;59:10401-6.

3. Reganold JP, Andrews PK, Reeve JR, Carpenter-Boggs L, Schadt CW, Alldredge JR, Ross CF, Davies NM, Zhou J. Fruit and soil quality of organic and conventional strawberry agroecosystmes. PLoS ONE 2010;5(9):e12346 [doi:10.1371/journal.pone.0012346].

4. Chang K. Stanford Scientists Cast Doubt on Advantages of Organic Mean and Produce. New York Times, 3 September 2012. Accessed at http://www.nytimes.com/2012/09/04/science/earth/study-questions-advantages-of-organic-meat-and-produce.html?_r=1 on 21 September 2012.

5. Brandt K, Leifert C, Sanderson R, Seal CJ. Agroecosystem management and nutritional quality of plant foods: The case of organic fruits and vegetables. Critical Reviews in Plant Science 2011;30:177-97 [doi:10.1080/07352689.2011.554417].

TO THE EDITOR: Smith-Spangler and colleagues’ laudable goal was to give “information that people can use to make their own decisions based on their level of concern about pesticides, their budget, and other considerations” (1). Unfortunately, the authors’ reporting went badly awry about the “risks” (incidences) of pesticide residues and antibiotic-resistant bacteria. Their Discussion greatly understates their findings: “…conventional produce has a 30% higher risk for pesticide contamination than organic produce” (2). The Stanford School of Medicine press release (1) has a similar understatement, and both miscues appeared in most major news stories.

     In usual terminology the actual “risk” (incidence) in conventional foods was over 5 times higher, not 30% higher. The authors found a 38% incidence in conventional foods compared to only 7% incidence in organic foods. Their “30%” comes from their unfamiliar “Risk Difference” metric, calculated as (approximately) 38% − 7%. In their Discussion and press release, they drop the word “difference” and make statements certain to be misinterpreted: “30% higher risk…in conventional foods.”

     Regarding antibiotic-resistant bacteria in chicken and pork, Fig. 5 shows a "Risk Difference" of 33%, but in the Results section, “difference" is again dropped: “The risk for isolating bacteria resistant to 3 or more antibiotics was 33% higher among conventional chicken and pork than organic alternatives.” Not so. Unfortunately, the authors did not report the separate “risks” (incidences) for conventional and organic meats, but Fig. 5 suggests incidences of about 49% for conventional chicken and pork and 16% for organic (a “difference” of 33%). Thus the actual incidence was about 3 times higher in the conventional meats, not 33% higher as stated by the authors and duly reported by the Associated Press, Reuters, and others.

     The authors were careless and misleading in their reporting, with the predictable result that many journal readers and certainly the public did not receive accurate information for making their own decisions about pesticides and bacteria.

Donald R. Davis, PhD

320 Jackson Hill St. Apt. 243

Houston, TX 77007

References

1. Brandt M. Little evidence of health benefits from organic foods, Stanford study finds. Stanford, CA. Stanford School of Medicine, 3 Sept. 2012. Accessed at http://med.stanford.edu/ism/2012/september/organic.html on 14 Sept. 2012.

2. Smith-Spangler C, Brandeau ML, Hunter GE, Bavinger JC, Pearson M, Eschbach PJ, et al. Are organic foods safer or healthier than conventional alternatives? A systematic review. Ann Intern Med. 2012;157:348-366.

Disclosures: The author is a retired research scientist from the University of Texas and a consultant to The Organic Center, a not-for-profit research and education organization focused on organic food and farming.

TO THE EDITOR: Smith-Spangler et al. (1) chose the frequency of residues in conventional versus organic food as their basic metric to assess pesticide-related health benefits of organic food. This choice was inappropriate; frequency of residues is not an indicator of risk. Pesticide dietary risk is a function of many factors including the number of residues, their levels, and pesticide toxicity. An appropriate assessment would use the extensive, high quality data on pesticide residues in organic and conventional food from the U.S. Department of Agriculture’s “Pesticide Data Program” (2). This publicly accessible data allows comparisons of residue frequency, levels, and chronic risks. Using a measure of chronic pesticide risks based on EPA assessment methods, I calculate that the average pesticide risk level is about 90% lower in the organic vs. conventional fruit and vegetable samples tested in 2010, and the average odds ratio is 32.7 (ave. conventional risk/ave. organic risk), indicative of a highly significant difference.The authors used six, multiple-food residue studies in the meta-analysis. The studies are highly variable in quality, foods tested, analytical methods, and limits of detection. Five of them tested European food, and the only U.S. study analyzed data well over 10 years old. Given how dramatically pesticide use patterns and residues in food have changed since passage of the Food Quality Protection Act in 1996 (3), the pesticide “risk” findings in Smith-Spangler are essentially irrelevant to what American consumers are now facing.Organic farming largely eliminates the human health risks associated with both pesticides in food and animal agriculture’s use of antibiotics. I agree with Smith-Spangler et al. that more science is needed to fully quantify these risks, but there is strong evidence (4, 5) that dramatically reducing them would be of clinical significance.Finally, the “Risk Difference” (RD) measure is misleading. The team reports a −30% pesticide RD between organic and conventional foods (approximately 7% of organic samples positive minus 38% conventional). As presented, this finding was often understandably interpreted as referring to pesticide health risk. It does not. Plus, the construct of this metric obscures the real magnitude of differences. If 90% of conventional and 60% of organic samples had residues (instead of 38% and 7%), one would get the same RD of −30%, which is equivalent to a modest 34% reduction in residue frequency. The authors’ reported −30% RD is actually an 82% reduction in frequency.

Charles Benbrook, PhD Research Professor Washington State [email protected]

References

1. Smith-Spangler C, Brandeau ML, Hunter GE, Bavinger JC, Pearson M, Eschbach PJ, et al. Are organic foods safer or healthier than conventional alternatives? A systematic review. Ann Intern Med. 2012;157:348-366.

2. Pesticide Data Program, Agricultural Marketing Service, U.S. Department of Agriculture, 2010 program summary and multiple years

3. “Food Quality Protection Act” of 1996, U.S. EPA, http://www.epa.gov/opp00001/regulating/laws/fqpa/

4. Bellinger, David. 2012. A Strategy for Comparing the Contributions of Environmental Chemicals and Other Risk Factors to Neurodevelopment of Children, Environmental Health Perspectives (Vol. 20 (4): pages 501-507).

5. Looft T, Johnson TA, Allen HK, Bayles DO, Alt DP, Stedtfeld RD, Sul WJ, Stedtfeld TM, Chai B, Cole JR, Hashsham SA, Tiedje JM, Stanton TB. 2012. In-feed antibiotic effects on the swine intestinal microbiome, Proceedings of the National Academy of Sciences, Vol. 109 (5): 1691-1696.

Disclosures: I served as the Chief Scientist of The Organic Center from 2005 to June 2012, a non-profit organization focusing on the health benefits of organic food and farming.

TO THE EDITOR: The meta-analysis by Smith-Spangler et al. (1) found significant differences for only one of six groups of secondary metabolites in plant foods (Table 1), contrasting with a previously published meta-analysis, which showed significant differences for four of six groups (2). Several aspects of the methodology used seem insufficiently justified or inconsistent, affecting the quality of the meta-analysis. Five issues require clarification: 1. The meta-analysis method used is only valid if appropriate sample sizes are used for the calculations (1). Recognized standards for good practice (3) emphasize that “coding of data from the articles” should be “specified and objective.” However, no procedure for allocation of sample sizes was presented (1), and a comparison with the design descriptions in the papers shows no consistent patterns. For example, for Ref. 217 the “sample sizes” in supplement 4 (1) correspond with the numbers of independently analyzed subsamples (constituting the reported averages and standard deviations, so probably appropriate), in Ref. 260 the “sample size” (1) is the total number of subsamples from all plots and years, even though results were reported separately per year, while for Ref. 240 and Ref. 151 the “sample sizes” (1) are the total numbers collected of tomatoes and leaves, respectively, irrespectively of how many of these had been analyzed together. 2. Explanations should be provided for the choices and definitions of groups of secondary metabolites, in particular why the flavonols were divided into 3 separate groups (increasing the type 2 error), and why most relevant groups other than “Total phenols” and “ß-carotene” (e.g. phenolic acids) were excluded. 3. Most detail about exclusion of studies and data extraction is missing, making it impossible to reproduce the analysis or detect errors. Best practice in terms of transparency would have been to publish these details online, as in (2) and (4).4. Despite stating that only peer-reviewed, English-language studies were eligible for inclusion (in contrast to recommendations in [3]), the authors included data from Ref. 143 (not peer-reviewed) and Ref. 239 (in Polish) in the calculations (1).5. It is unclear why data on secondary metabolites from 14 included studies (Supplement 4 [1]) were not included in the calculations (Supplement 2 [1]).Clear explanation and justification of procedures plus publication of the full extracted dataset are necessary to assess why the results of the two meta-analyses (1, 2) differ and which methodology is best suited for this type of data.

References

1. Smith-Spangler C, Brandeau LM, Hunter GE, Bavinger JC, Pearson M. Are Organic Foods Safer or Healthier Than Conventional Alternatives? A Systematic Review. Ann Intern Med. 2012;157:348-66.

2. Brandt K, Leifert C, Sanderson R, Seal CJ. Agroecosystem management and nutritional quality of plant foods: the case of organic fruits and vegetables. Crit Rev Plant Sci. 2011 2011/04/22;30(1-2):177-97.

3. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000; 283:2008-12.

4. Seufert V, Ramankutty N, Foley JA. Comparing the yields of organic and conventional agriculture. Nature. 2012;485(7397):229-32.

Disclosures: Potential conflicts of interests: Some industry-related funding to my institution for my research (£12,250) is for projects related to organic food (e.g. from Soil Association), while a larger portion (£145,197) is for projects not related to organic food, e.g. from Danone and Asda (part of Wal-Mart). The majority of my public grant income is also not related to organic food. I am corresponding author on the Brandt et al. paper (2).

I read with interest the recent systematic review by Smith-Spangler and coworkers (1). I would like to present an alternative view on the results of Smith-Spangler’s metaanalyses on nutrient content in organic vs. conventional crops. I argue that Smith-Spangler actually has found interesting indications for differential nutrient content.According to the article’s table 1, the concentrations of ascorbic acid, calcium, phosphorus, magnesium, quercetin, kaempferol, and total phenols are on average significantly higher in organic than in conventional crops (p<0.05). That is, the standardized mean difference (SMD) is positive and the value 0 is not within the 95% CI for the SMD. This amounts to 7 out of 14 tested nutrients or nutrient categories. Where overlapping, these results are largely consistent with one recent review (2) but only partly with another one (3).Smith-Spangler rightfully acknowledges that a large number of comparisons (22 for crops) have been performed. With a significance level of p=0.05, one can expect 1/20 of the tested hypotheses (comparisons) to be falsely positive. Smith-Spangler uses the Sidak correction for multiple testing and reports accordingly adjusted p-values, leaving only phosphorus and total phenols significantly higher in organic crops. Smith-Spangler finally summarizes this as a lack of strong evidence for differential nutrient content.One could make the point that a systematic review should focus on the estimation of effects, rather than on rigorously testing for them. Indeed, this is the view of the Cochrane Collaboration, renowned for their systematic reviews in health sciences. For this reason, they also state that “Adjustments for multiple tests are not routinely used in systematic reviews, and we do not recommend their use in general.” (4)In the present case, one would expect approximately 1 of the tested hypotheses on nutrient content in crops to be falsely positive. Accordingly, 6 out of 14 tested nutrients are probably true positives and therefore significantly higher in concentration in organic crops. In the light of this, the applied Sidak correction is likely obscuring real effects of the agricultural production system on the nutrient composition, as it decreases the list of significant differences to 2.It is my view that Smith-Spangler actually is presenting good indications that organic crops are richer in a number of nutrients, compared to conventional crops. I also believe that such differential nutrient content cannot be directly translated into differential health effects, because food consists of thousands of compounds rather than 14. As Smith-Spangler points out, only very few studies have actually attempted to measure human health outcomes of conventional vs. organic food consumption. In my eyes, Smith-Spangler’s findings may well serve as a justification and encouragement for conducting such research into health effects.

References:

(1) Smith-Spangler, C., et al., Are Organic Foods Safer or Healthier Than Conventional Alternatives? A Systematic Review. Annals of Internal Medicine, 2012. 157(5): p. 348-366.

(2) Brandt, K., et al., Agroecosystem Management and Nutritional Quality of Plant Foods: The Case of Organic Fruits and Vegetables. Critical Reviews in Plant Sciences, 2011. 30(1-2): p. 177-197.

(3) Dangour, A.D., et al., Nutritional quality of organic foods: a systematic review. Am J Clin Nutr, 2009. 90(3): p. 680-685.

(4) Higgins JPT, Deeks JJ, Altman DG (editors). Chapter 16: Special topics in statistics. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

Disclosures: some research grant money related to organic food (2008, ca 116000 Euro)

In Figure 2 of a recent article (1), the conventional number for the study by Tasiopoulou et al should be 2342, not 3242.

The last sentence of the "Pesticide Contamination" section on page 354 should read as follows: Differences in prevalence of contamination exceeding maximum allowed limits were small among the other 2 studies (6% [60 of 1048 samples] for organic vs. 2% [179 of 2237 samples] for conventional, and 1% [1 of 266 samples] for organic vs. 1% [36 of 324 samples] for conventional).

In Table 1 and Supplement 2 of a recent article (1), the word "flavanols" should be "flavanoids."

In Figure 5, the word "produce" in the figure labels should be "products."

Reference

1. Smith-Spangler C, Brandeau ML, Hunter GE, Bavinger JC, Pearson M, Eschbach PJ, et al. Are organic foods safer or healthier than conventional alternatives? A systematic review. Ann Intern Med. 2012;157:348-66.

Authors’ Response to Dr. Brandt’s Comments about Methods and Reporting,

To The Editor:

Dr. Brandt raises several questions regarding differences in the methods that we used in our analysis [1] and those of her and her colleagues published in Critical Reviews in Plant Sciences (2). In their review, Brandt et al. found statistically significantly higher levels of 1) total phenolics, 2) “other (plant) defense compounds” (tannins, alkaloids, chalcones, stilbenes, flavanones and flavanols, hop acids, coumarins, and aurones), 3) flavones and flavanols, and 4) vitamin C. Differences between the remaining groups examined by Brandt et al., carotenes and “other non-defense compounds” (anthocyanins, tocopherols, and volatiles), were not significant between organic and conventional samples.

Some of our results were similar to those of Brandt et al.: We both found organic produce to have significantly higher levels of total phenols but no differences in alpha-tocopherol or beta-carotene [1]. We did not examine “other plant defense compounds” as a group.

However, whereas we found no significant difference in the vitamin C content of organic and conventional foods and significant heterogeneity in the summary effect estimate, Brandt et al. found a small but statistically significantly higher level of vitamin C among organic produce compared to conventional alternatives [2].

Our study [1] differed substantially from that of Brandt et al. [2] by formally considering both within-study and between-study heterogeneity (variability) and included a correction for multiple comparisons.

First, our study took into account within-study variation by including the variance reported by each of the studies in summary effect calculations whereas Brandt et al. did not appear to include this information in their calculations. When information about within study variance is not included, more weight is given to studies with high variability. Also, note that of all the studies comparing vitamin C in organic and conventional produce (including studies reporting results incompletely so that statistical tests could not be conducted and they could not be included in summary effect calculation, see Table 1 [1]) in only 23 of the 113 comparison pairs were there statistically significant differences favoring organic produce. In 78 comparisons, differences between organic and conventional were not statistically significant.

Second, in contrast to Brandt et al., we reported heterogeneity statistics for all findings (which examined between-study variability), conducted sensitivity analyses when possible to explore sources of heterogeneity (e.g., testing method, study design, organic standard applied), and conducted sensitivity analyses to detect influential studies [1] —all standard practices for the conduct of a meta-analysis.

Third, our analyses included a correction for multiple comparisons. Multiple comparisons becomes a problem when one is looking at a large number of outcomes, particularly when these findings come from the same samples, which are not independent but instead correlated, as is the case in our review of the literature on organic and conventional foods [1]. Multiple comparisons increase the likelihood of finding significant differences when none exist (false positives). Additionally, we included each study only once in summary effect calculations in order to avoid correlation effects (i.e., findings from the same study but different years are likely to be correlated) in contrast to Brandt et al. who assumed samples from the same study but different years or seasons are independent.

Finally, our choice of outcomes and grouping of outcomes reflects our different perspectives. We were principally interested in the outcomes that consumers use to inform their shopping decisions and those that were reported frequently in the literature (e.g., ascorbic acid, alpha tocopherol). We would have liked to examine flavanol content, which has been implicated in the health benefits of chocolate [3], though too few studies reported on this group. Instead, we assessed a larger group of which flavanols are a part (flavanoids) and did not find differences between organic and conventional produce to be significant [1].

Dr. Brandt has suggested that we have not provided sufficiently detailed information about our methods or results. We disagree. As described in our methods, we extracted the data (e.g., means, variances, sample sizes) from each study and used standard methods for combining results using a commercially available software package, Comprehensive Meta-analysis, version 2 (Biostat, Englewood, New Jersey). We provide more than 70 pages of detailed supplemental material about our methods and included studies (available at annals.org). For example, Supplement 4 describes each included study in detail, including sample sizes, outcomes measured, and a summary of key findings. Additionally, we provide much of the raw data on key outcomes such as risk of contamination in the published figures (e.g., Figure 2-5).

We also describe our inclusion and exclusion criteria in our methods and have even listed the studies that reported results but did not report results completely such that they could be included in statistical calculations (see Supplement 2) [1]. For example, the 14 studies mentioned by Dr. Brandt did not report standard deviations or variances so we could not include them in summary calculations (as described in the Methods section, page 349) though the results of these studies are summarized qualitatively in Table 1 and Supplement 4. Furthermore, Ref 239 (mentioned by Dr. Brandt in her comments) included a summary with data in English (vitamin C) and it is our understanding that Ref 143 was peer reviewed (included in vitamin C, A, and quercetin analyses) so we included these articles in our review. However, as with all outcomes, we conducted sensitivity analyses in which each study was removed from the analysis one at a time to evaluate if any one study substantially changes the reported findings. Removal of Ref 143 or Ref 239 did not alter our findings.

A final note about our statistical methods: our paper underwent an extensive review process, including review by a statistical editor at the Annals of Internal Medicine and discussion by their statistical board to assure that our approach adhered to standard methods of conducting and reporting meta-analyses.

We hope these comments highlight the key differences between our analysis and that of Brandt et al. and thereby illuminate the sources of some of the differences in our results.

Crystal Smith-Spangler, MD, MS, Margaret Brandeau, PhD, Ingram Olkin, PhD, Dena Bravata, MD, MS

References

1. Smith-Spangler C, Brandeau ML, Hunter GE, Bavinger JC, Pearson M, Eschbach PJ, et al. Are organic foods safer or healthier than conventional alternatives? A systematic review. Ann Intern Med. 2012;157:348-366.

2. Brandt K, Leifert C, Sanderson R, Seal CJ. Agroecosystem management and nutritional quality of plant foods: the case of organic fruits and vegetables. Crit Rev Plant Sci. 2011;30(1-2):177-97.

3. Ried K, Sullivan TR, Fakler P, Frank OR, Stocks NP. Effect of cocoa on blood pressure. Cochrane Database Syst Rev. 2012;8:CD008893.

Response to Comments by Dr. Andrews Regarding Methods:

To the Editor:

We agree with Dr. Andrews that genetic and environmental factors influence plant metabolism and hence the accumulation of phytonutrients. However, consumers are not generally given information about genetics or environmental factors when they are purchasing their food and instead, only have information about whether the food was organic or conventionally produced. Hence, to reflect the choices of consumers, we felt it was fair to include all data comparing organic and conventional.

We regret that the study of organic and conventional strawberries by Reganold, Andrews, and colleagues [1] was not included in our analyses [2] of organic and conventional foods as their study was erroneously coded it as soil study. However, upon additional review, we find that this study [1] has data that would meet criteria for inclusion. We point out that our review [2] did find significantly higher levels of total phenols among organic produce, similar to Reganold et al.’s findings of higher levels among organic compared with conventional strawberries [1]. However, in contrast to Reganold et al.’s study, our review did not examine antioxidant capacity as an outcome and we did not find significant differences in Vitamin C levels between organic and conventional produce [2]. Given that our review included 31 other studies in our summary effect calculation for Vitamin C (and found a large p-value; p=0.48, Table 1 [2]), it is highly unlikely that inclusion of one more study would have substantially altered results.

References

1. Reganold JP, Andrews PK, Reeve JR, Carpenter-Boggs L, Schadt CW, Alldredge JR, et al. Fruit and soil quality of organic and conventional strawberry agroecosystems.[Erratum appears in PLoS One. 2010;5(10). doi: 10.1371/annotation/1eefd0a4-77af-4f48-98c3-2c5696ca9e7a]. PLoS ONE [Electronic Resource]. 2010;5(9):2010.

2. Smith-Spangler C, Brandeau ML, Hunter GE, Bavinger JC, Pearson M, Eschbach PJ, et al. Are organic foods safer or healthier than conventional alternatives? A systematic review. Ann Intern Med. 2012;157:348-366. 

Response to Comments and Questions about the Purpose of the Study, Funding, Reporting, and Methods

To The Editor:

There has been considerable attention from the media, medical and agricultural research communities, and the general public since the publication of our article, “Are organic foods safer or healthier than organic alternatives?” [1] including several comments posted on the Annals of Internal Medicine website. Many of these inquiries posed similar questions and we would like to share our responses in summary for interested readers.

Purpose of the study: We were interested in the evidence of whether organic foods are more nutritious, safer, or healthier than conventional alternatives. Although consumers are likely most interested in whether consumption of organic food improves health or reduces disease, we only identified 17 studies of humans consuming organic and conventional foods and only 3 of these studies examined a very small number of clinical outcomes [1]. Given this limited literature, we next examined indirect measures of health and disease (e.g., nutrient levels and risk of contamination among organic and conventional products) to understand differences between organic and conventional foods that may influence health and disease.

Our comparison of pesticide contamination or antibiotic resistant bacteria among organic and conventional foods, for example, was not designed to, and is not able to, assess the safety of current pesticide levels in produce or antibiotic resistant bacteria on meat products but instead to provide information to consumers about differences in these outcomes between organic and conventional products. Consumers vary in terms of the outcomes of greatest importance to them: for example, some consumers are interested in whether certain foods are more nutritious than others (e.g., contain higher vitamin or mineral levels), whereas others are more concerned about food safety issues (e.g., bacterial or pesticide contamination). Thus, we presented the evidence for a variety of outcomes of interest so that individual consumers can make preference-sensitive decisions.

Furthermore, as we mention in our publication [1], there are numerous valid reasons why consumers might chose organic over conventional foods including concerns about the environment, animal welfare, farm worker health, taste, and cost. Our study did not address any of these issues; instead we sought to synthesize the evidence on nutrient levels, contamination, and known health effects of the decision to consume organic versus conventional foods.

Project funding: Study authors did not receive funding from any agricultural, chemical, or food organization or business for this project and authors do not have other work funded by these organizations or businesses. Ms. Pearson (who was an undergraduate at the time of the project) was supported by a Stanford Undergraduate Research Grant for one summer. Dr. Smith-Spangler (who was a post-doctoral fellow during the time of this project) was supported by the VA Physician Post-Residency Fellowship, a research training program. This study was not supported by any other funding mechanisms including grants or contracts.

Interested readers may examine the ICMJE Disclosure of Potential Conflicts of Interest statements that are available from all authors at www.annals.org under author and article information. To summarize these statements by the authors, neither the study authors nor Stanford University or any other institution affiliated with the study authors received a grant, a consulting fee or honorarium, support for travel to meetings for the study or other purposes, fees for participation in activities, payment for writing or reviewing the manuscript, provision of writing assistance/equipment/administrative support or other types of support from a third party to support any aspect of the submitted work. Furthermore, no authors have any financial relationships (e.g. board membership, consultancy, employment, expert testimony, grants/grants pending, payment for lectures, payment for manuscript preparation) with any entities (e.g. agricultural, chemical, or food organizations or businesses) that could be perceived to influence our published work.

Methodological questions: The three most common questions about our methods relate to 1) our use of the absolute risk difference when comparing pesticide residues in organic and conventional produce, 2) our presentation of p values and consideration of statistical significance in light of multiple comparisons, and 3) how our methods differ from those of Dr. Brandt and colleagues [2]. We address the first two points here, and separately provide a detailed discussion comparing our work to that of Brandt et al. We also note that our paper was reviewed by a statistical editor at the Annals of Internal Medicine and discussed by their statistical board to assure that our approach adhered to standard methods of conducting and reporting meta-analyses.

Absolute risk difference: We have been asked whether our choice of the absolute difference in risk of exposure to any detectable pesticide residue is an appropriate effect size. We found a 30% reduction in absolute risk of contamination with any detectable pesticide residues among organic produce compared with conventional produce [1]—a substantial reduction in risk. We disagree with those who suggest that our use of absolute risk difference is misleading because we clearly describe the use of absolute risk difference in both the methods and results. Furthermore, in epidemiology, it is considered to be best practice to report absolute risk differences rather than relative risk differences, particularly when events are uncommon as was the case with the included evidence. Indeed, for this reason, we believe it would have been misleading to use measures of relative risk reduction. Additionally, we report absolute risk differences throughout the paper, not just for pesticide contamination but also for contamination with bacteria and antibiotic-resistant bacteria, for similar reasons.

We are aware that other outcomes related to pesticide contamination in foods are of interest to consumers including the magnitude of levels of pesticide residues and exposure to multiple pesticide residues. However, we only found 9 studies comparing pesticide outcomes (and 3 of these were single-food studies). Data on pesticide levels and contamination with multiple pesticides were reported inconsistently among these studies; thus, we could not summarize results on absolute pesticide levels or contamination with multiple pesticides. However, 3 studies did report risk of contamination exceeding maximum allowed limits, which we assessed and report in our publication.

Statistical significance and multiple comparisons. We present confidence intervals and p values for all reported outcomes [1]. The p values presented are those after correction for multiple comparisons. Multiple comparisons increase the likelihood of finding significant differences when none exist (false positives). The problem of multiple comparisons occurs when one examines a large number of outcomes, particularly when these outcomes come from the same samples, which are not independent but instead correlated, as is the case in our review of the literature on organic and conventional foods [1]. We believe that our conservative approach to calculating the p values is justified on the basis of the multiple comparisons and substantial heterogeneity among studies. Moreover, there was some concern that if we had presented the unadjusted p values, it would be confusing to the reader accustomed to a threshold of p=0.05 as the standard measure of statistical significance that a given effect was considered not meeting statistical significance at that level.

Reporting on outcomes of interest: In the text, tables, and figures of our publication [1] we present detailed information about the abstracted data and the analyses we performed so that interested parties can interpret the data for themselves and to facilitate replication of our results. In addition to the data provided with the main text, we provide more than 70 pages of detailed supplemental material about our methods and included studies (available at annals.org). We have been criticized for deemphasizing the importance of some key results of interest—most notably the reduction in absolute risk of contamination with any detectable pesticide residues and antibiotic-resistant bacteria among organic products. These are some of the most important findings of our study and we highlighted the substantial reduction in absolute risk of exposure to any detectable pesticide residues and to multidrug resistant bacteria in selected organic foods in the abstract, discussion, and multiple figures [1].

Heterogeneity: As we point out in our discussion, there are multiple sources of heterogeneity (variability) among the included studies that should be kept in mind when interpreting our results [1]. First, study methods (e.g., organic standard applied) varied considerably among the included articles by both geography (included studies were from many different countries) and over time (the included articles span many years, over which time organic standards have evolved). Second, the foods that were evaluated (e.g., the specific cultivars) could have significantly affected the measured outcomes (e.g., nutrients). Third, environmental factors such as differences in harvesting, storage, and processing, all can affect the outcomes of interest. Finally, farming practices can vary greatly among organic farmers (3). Any of these sources of variability could have obscured the ability to detect true differences between organic and conventional foods. However, consumers generally do not have specific information about the exact practices followed by farmers such as harvest date, weather conditions, fertilizer type/concentration/schedule, cultivar or breed of animal. Thus, it seems appropriate to combine studies with these types of heterogeneity since they reflect the information available to consumers faced only with an organic or conventional label to guide their consumption decision.

References

1. Smith-Spangler C, Brandeau ML, Hunter GE, Bavinger JC, Pearson M, Eschbach PJ, et al. Are organic foods safer or healthier than conventional alternatives? A systematic review. Ann Intern Med. 2012;157:348-366.

2. Brandt K, Leifert C, Sanderson R, Seal CJ. Agroecosystem management and nutritional quality of plant foods: the case of organic fruits and vegetables. Crit Rev Plant Sci. 2011;30(1-2):177-97.

3. Knoblauch W, Brown R, Braster M. Organic field crop production: a review of theeconomic literature. Agriculture Experimental Research. Ithaca, NY: Department of Agricultural Economics, Cornell University; 1990.