State of Science :: Commentaries
Response to ACSH Report on the TOC Nutrient Content SSR
Author(s): Charles Benbrook
In March 2008 The Organic Center released a report entitled "New Evidence Confirms the Nutritional Superiority of Plant-Based Organic Foods." This report analyzes and integrates the results of published studies comparing the nutrient content of conventional and organic food.
We used a set of objective criteria to identify 97 studies that were well designed and used reliable analytical methods. We analyzed the differences reported in these studies in the concentrations of over three-dozen nutrients. There were an adequate number of direct comparisons to reach judgments regarding differences in the case of 10 nutrients, plus nitrates.
Across these 11 components in food, the organic samples were, on average 25% more nutrient dense than the conventional samples.
The American Council for Science and Health (ACSH) report by Dr. Joseph Rosen seems to accept, and does not argue with the factors we identified to screen published nutrient content studies for scientific validity. He does not address, and hence seems to accept the 17 criteria we specified for choosing the valid matched pairs of foods from the 97 studies we identified. He singles out, and agrees with our classification of the Asami et al. 2003 work at University of California-Davis as poorly designed.
But as he goes through the various sections of our report, he criticizes us for "cherry-picking" results, excluding matched pairs where the results favor conventional food. That is not what we did. He misrepresents our methodology and offers inconsistent and illogical suggestions to correct what he perceives as "bias" in our selection of the matched pairs of foods that were then used to compare nutrient concentrations.
We applied the screening method and selection criteria consistently, and in fact eliminated more results favoring the organic food in a matched pair than the conventional food. Without our screening methods and criteria, the nutritional advantage of organic food would have been greater.
While he never disagrees with the criteria we used to screen the studies and select the matched pairs, he adds several new criteria. He criticizes us for including the results of a kiwi study where the scientists measured the nutrient content of the whole fruit, including the peel. He argues people don't usually eat the peel, so we should have dropped that matched pair. But whole kiwis are, in fact, used and consumed in several dishes. On the basis of this kiwi study and criticism, he imposes a new criterion in his re-analysis -- valid studies are those that report results for just the edible portion of a crop.
Rosen argues that any study reporting a difference of 20% or less is not meaningful, and on this basis, knocks out about 2/3 of the matched pairs. This clearly biases the results of his re-analysis to exceptional cases where differences in nutrient levels over 20% are reported.
The ACSH report argues that nitrate consumption is actually healthy, and instead of counting the much higher level of nitrates in conventional food as an advantage for organic, Rosen counts it as an advantage for conventional food. This change in classification, clearly not backed by the preponderance of evidence or views in the scientific community, accounts for most of the quantitative difference in his "re-analysis" of our summary statistics.
We reported, on average across 11 nutrients, that nutrient levels were 25% higher in the organic samples in our matched pairs; after knocking out 2/3 of the matched pairs, and changing the classification of nitrates from "bad" to "good," Rosen calculates a 2% nutrient density advantage for the conventional samples in the remaining matched pairs.
Rosen writes that since a consumer worried about Vitamin C intake can get more than enough with a $0.10 daily supplement pill, the Vitamin C and ascorbic acid matched pairs should be disregarded. However, Vitamin C and ascorbic acid are routinely measured in nutrient content studies and daily intakes from food are inadequate for millions of Americans. Furthermore, he states that we erred by combining the results of studies that compared kaempferol levels in a matched pair, with studies that reported differences in the precursor to kaempferol. Because of the close relationship between levels of the precursor of a nutrient and the levels of the nutrient, scientists often combine the results and report them under one category.
He also makes several technical points/criticisms for example, he writes that because the organic farmers in one study applied the National Organic Program/USDA-approved pesticide chitosan, the matched pairs from this study are biased because chitosan can stimulate production of kaempferol. He notes that both organic and conventional farmers use chitosan. He fails to acknowledge, however, that hundreds of conventional pesticides also impact phytonutrient levels in many different ways. Why single out just one chitosan??
The point of our review was to synthesize the reported results of published studies, based on the production systems the scientists adopted as representative of conventional and organic farming. We recognize many of the studies that passed our screens of scientific validity raise additional questions in need of further research, but our report presents an accurate and up to date synthesis of a large body of published science based on a clearly stated, transparent set of criteria. Our results are clear-cut, significant, and scientifically sound.
Our March 2008 report is based on published studies released through the fall of 2007. Since then, nearly a dozen new studies have come out, which we will include in subsequent analyses. As stated in the March 2008 report, we think the evidence is now sufficiently robust and consistent to reach some general conclusions about nutrient density in conventional and organic food, but accept that some organizations like the American Council for Science and Health, and individuals like Rosen will remain skeptical for many years to come, regardless of what the science shows.