Lies, damn lies and statistics
In the past 2 weeks there has been considerable press about a forthcoming article in the journal Archives of Environmental Contamination and Toxicology. This article “New Analysis of a Rat Feeding Study with a Genetically Modified Maize Reveals Signs of Hepatorenal Toxicity” by Gilles-Eric Séralini, Dominique Cellier and Joël Spiroux de Vendomois, purports to show that a genetically-modified corn causes damage to the livers and kidneys of rats and hence is likely to be dangerous to humans. The article has not yet been published, so unless you are a subscriber to the journal you cannot get a copy.
The article contains no new data. It has taken data already published in 2006 in the journal Food and Chemical Toxicology and re-analysed them. In doing so, the authors of the new paper have suggested they found important differences missed the first time around.
This is where the statistical “lies” come in. When scientists want to compare the effects of two treatments they use statistics because it is impossible to test the treatment on the whole population. If the treatment effect differs by enough from the control, they describe it as significantly different. Often a probability value of 5% is used. You can think of this as the scientist betting that there is only a 5% chance that the difference is not real. In statistics these are known as Type 1 errors.
It is therefore obvious that if a large number of comparisons are made between two items that the chance of a Type 1 error is increased. This is the problem of multiple comparisons in statistics. By the time 100 different comparisons are made you are almost certain to find at least one difference at 5% even when none exists. For this reason, good scientists avoid falling into this trap, either by conducting a different type of statistical test or by reducing the probability value at which they describe something as significant.
Seralini and his co-workers in their paper reported 494 comparisons between rats fed GM food and rats fed conventional food. With this number of comparisons, you would expect to get 25 differences at a probability of 5% and 5 differences at a probability of 1% where no differences exist. Seralini reports 33 differences at 5% and 4 differences at 1%; almost exactly the number you would predict to occur through chance. This is an elementary problem in statistics and a practice that good scientists avoid. It also totally undermines any conclusions drawn from the analysis.
Intuitively, we understand that if something we eat is dangerous then eating more of it will be more dangerous, or at least no less dangerous. In science this is the dose response. The effects of a toxin will become more pronounced with more toxin. It is also intuitive that consuming a toxin for a longer time will also be more dangerous. The dose response underpins all toxicology. In the present study, rats were fed either 11% or 33% GM corn for 5 or 14 weeks. None of the differences reported by Seralini followed a dose response. The differences were either sporadic or were better with higher concentrations. Without a dose response, it is not possible to link any of the differences reported to consumption of the product.
As there is no dose response and the differences reported by Seralini are likely due to chance, this study does not support the notion that this GM corn is hazardous to the health of rats. This is the same conclusion that the 2006 paper came to. How then could Seralini get it so wrong? This could easily be dismissed as an argument between statisticians; however, the problems are so basic and so obvious that they show scientific incompetence, at best. It also raises questions about how the paper came to be published in the first place.
What is worse is that this bad science is being used to influence the World’s media and decision makers. Greenpeace, who funded the paper, is hawking these statistical lies around. Clearly they are having influence for just last week the Minister for Agriculture in Western Australia used this as support for a moratorium on GM crops.
Dr. Christopher Preston
Discipline of Plant & Food Science
University of Adelaide
Update: Study citation
New Analysis of a Rat Feeding Study with a Genetically Modified Maize Reveals Signs of
Hepatorenal Toxicity
Séralini, Gilles-Eric, Cellier, Dominique and de Vendomois, Joël Spiroux
Archives of Environmental Contamination and Toxicology 2007
10.1007/s00244-006-0149-5
Listed as Seralini GE, Cellier D, de Vendomois JS. New analysis of a rat feeding study with a genetically modified maize reveals signs of hepatorenal toxicity. Arch Environ Contam Toxicol. 2007 May;52(4):596-602. Epub 2007 Mar 13. PMID: 17356802 in Pubmed
Abstract Health risk assessment of genetically modified organisms (GMOs) cultivated for food
or feed is under debate throughout the world, and very little data have been published on mid- or long-term toxicological studies with mammals. One of these studies performed under the responsibility of Monsanto Company with a transgenic corn MON863 has been subjected to questions from regulatory reviewers in Europe, where it was finally approved in 2005. This
necessitated a new assessment of kidney pathological findings, and the results remained
controversial. An Appeal Court action in Germany (Münster) allowed public access in June 2005 to all the crude data from this 90-day rat-feeding study. We independently re-analyzed these data. Appropriate statistics were added, such as a multivariate analysis of the growth curves, and for biochemical parameters comparisons between GMO-treated rats and the controls fed with an equivalent normal diet, and separately with six reference diets with different compositions. We observed that after the consumption of MON863, rats showed slight but dose-related significant variations in growth for both sexes, resulting in 3.3% decrease in weight for males and 3.7% increase for females. Chemistry measurements reveal signs of hepatorenal toxicity, marked also by differential sensitivities in males and females. Triglycerides increased by 24–40% in females (either at week 14, dose 11% or at week 5, dose 33%, respectively); urine phosphorus and sodium excretions diminished in males by 31–35% (week 14, dose 33%) for the most important results significantly linked to the treatment in comparison to seven diets tested. Longer experiments are essential in order to indicate the real nature and extent of the possible pathology; with the present data it cannot be concluded that GM corn MON863 is a safe product. [Reanalysis of existing studies , funded by Greenpeace]
Acknowledgments. We thank Anne-Laure Afchain for her help in statistical analyses, and the CRIIGEN scientific and administrative councils for expertise, and initiating judiciary actions by the former French minister of environment, Corinne Lepage, to obtain the data.
We also thank Frederique Baudoin for secretarial assistance, and Dr. Brian John and Ian Panton for advising on the English revision of the manuscript. This work was supported by Greenpeace Germany who, in June 2005, won the Appeal Court action against Monsanto,
who wanted to keep the data confidential. We acknowledge the French Ministry of Research and the member of Parliament FranÅois Grosdidier for a contract to study health assessments of GMOs, as well as the support of Carrefour Group, Quality, Responsibility and
Risk Management.
Update 29th June 2007
- The European food safety agency EFSA has just looked at the study, by a French group led by Gilles-Eric Seralini and concluded
The statistical analysis made by the authors of the paper did not take into account certain important statistical considerations. The assumptions underlying the statistical methodology employed by the authors led to misleading results.
Further update 17 July 2007.
EFSA considers that the paper does not present a sound scientific justification in order to question the safety of MON 863 maize.
A further GMO Pundit post detailing the context of Seralini study - French Greenpeace and environmentalist misinformation explained by a French CNRS director
Update 19 Dec 2007
EFSA response now published as
Report of an expert panel on the reanalysis by Seralini et al. (2007) of a 90-day study conducted by Monsanto in support of the safety of a genetically modified corn variety (MON 863) FOOD AND CHEMICAL TOXICOLOGY, 45 (11): 2073-2085 NOV 2007
Doull, J; Gaylor, D; Greim, H; Lovell, DP; Lynch, B; Munro, IC
Abstract:
MON 863, a genetically engineered corn variety that contains the gene for modified Bacillus thuringiensis Cry3Bb1 protein to protect against corn rootworm, was tested in a 90-day toxicity study as part of the process to gain regulatory approval. This study was reanalyzed by Seralini et al. who contended that the study showed possible hepatorenal effects of MON 863. An Expert Panel was convened to assess the original study results as analyzed by the Monsanto Company and the reanalysis conducted by Seralini et al. The Expert Panel concludes that the Seralini et al. reanalysis provided no evidence to indicate that MON 863 was associated with adverse effects in the 90-day rat study. In each case, statistical findings reported by both Monsanto and Seralini et al. were considered to be unrelated to treatment or of no biological or clinical importance because they failed to demonstrate a dose-response relationship, reproducibility over time, association with other relevant chang! es (e.g., histopathology), occurrence in both sexes, difference outside the normal range of variation, or biological plausibility with respect to cause-and-effect. The Seralini et al. reanalysis does not advance any new scientific data to indicate that MON 863 caused adverse effects in the 90-day rat study. (C) 2007 Elsevier Ltd. All rights reserved.
Labels: Crop science, Ethics, Safety and Regulations, Statistical interpretation
posted by GMO Pundit @ 7:32 AM
3 comments
links to this post
3 Comments:
The paper on which Preston comments is truly awful, the kind that my stats prof at Berkeley, Steve Selvin, used as an example in class as to how we were NOT to do stats!
In lay terms, the problem here can be illustrated by tossing a set of 5 coins. Getting all 5 to fall as heads is unusual, only once in every 32 tosses by chance alone. So let's say you do 40 tosses and one comes up all heads. Is that "magic" or a "significant" result in the statistical sense? Of course not; at least one was expected! That's in effect what Seralini and friends have done. They kept running tests until, by chance, they found interesting results.
And gm companies are of course completely above board and never 'fix' stats either! yeh right!
The lesson to be learned here is that neither side of the debate can be trusted. Just as Greenpeace funds only the scientists that will produce the results they want to hear, corporations like Monsanto fund only the scientists that produce results that serve their own interests (i.e.mega-profits through global control of agriculture).
Any scientist worth his/her salt will tell you that scientific method is never able to conclusively prove a theory to be true, or false for that matter. Science is not about discovering the truth it is about constructing the best possible theory. And statistics is a fetishistic object in that it serves only to veil the truth; hence the statement that there are truth, lies, and statistics.
So where does that leave the layperson when it comes to deciding who to believe? My suggestion is as follows: By all means consider the results of the research. But more importantly - consider the vested interests of those funding and doing the research. As far as this particular debate is concerned, of paramount importance is the consideration of the ethics,or rather the lack of ethics of corporations like Monsanto. I suggest readers look into the business practices of Monsanto (which are highly unethical). Patenting laws allow Monsanto to put farmers out of business when their crops inevitably become contaminated by neighboring fields of GM crops. The same applies to livestock. And Monsanto certainly have put farmers out of business. A recent documentary on Monsanto also revealed that their goal is to eventually take global control of farming industry - which is why I now boycott Monsanto products.
It has been said politicians are the enemy of the people. Today, it is more accurate to say corporations are the enemy of people since it is the corporations that lobby and bribe the politicians to introduce laws and policies that allow them to exploit people and trample over those that present an obstacle in their pursuit of ever larger profit margins. GM products are more about power and profit than feeding the world.
Post a Comment
Links to this post:
Create a Link
<< Home