select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
This analysis accompanied a roundup which can be viewed here.
‘Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize’, Food and Chemical Toxicology 19 September 2012
The paper does not prove the claim that rats fed on a diet containing NK603 Roundup tolerant GM maize died earlier than rats fed on a standard diet.
Looking at the graphs of mortality for females (Fig 1), death might appear to be earlier with a GM diet than a standard diet; however this has not been proven statistically. Mortality is broadly similar for males with a GM diet and a standard diet. Similar comments apply to pathological findings.
It is evident that some treated groups have lower death rates / tumour rates than the comparable controls. This is not reported in the abstract.
There is no consistent dose-trend – if there were an effect, we would expect to see increases (e.g. in deaths) from 0 to 11 to 22 to 33. In contrast – in males, 33 (and C) have the lowest numbers of deaths.
“In females, all treated groups died 2–3 times more than controls, and more rapidly. This difference was visible in 3 male groups fed GMOs.” – this statement has not been subjected to standard methods of statistical analysis for survival time. The phrase “died 2–3 times more than controls” is based on exceptionally small numbers.
The authors suggest a threshold – this rarely occurs in practice. We would expect greatest mortality/ toxicity at the highest dose in a well-designed study. With small numbers as in this study we would expect to see a general trend of mortality increasing with dose.
It seems likely that the numbers in each group are too small for standard methods of statistical analysis to find significant effects on mortality or pathological findings.
There are virtually no p-values presented. The group sizes are small. This should be interpreted with extreme caution.
It is notable that the figures do not present deaths in the control group in a similar manner (no step graph for controls). This makes it more difficult to compare the other groups with the controls.
There are many treated groups, and a number of parameters. There is obvious potential for selected reporting, selection of methods etc. In such small groups, with so many parameters this is a big issue. This issue is amplified in the abstract and further in the press release. Strong statements are issued without sufficient backing / explanation.
Deaths are compared to the control mean (for males and females). Due to the distribution of deaths (most deaths occur in old age), this is almost bound to exclude the large majority of deaths in the control groups. The 2 or 3 deaths in the control group is determined by their methods, but is inappropriately presented as a true observation.
No
‘Before the headlines’ is a service provided to the SMC by volunteer statisticians: members of the Royal Statistical Society (RSS), Statisticians in the Pharmaceutical Industry (PSI) and experienced statisticians in academia and research. A list of contributors, including affiliations, is available here.