Title, Date of Publication & Journal

A long-term toxicology study on pigs fed a combined genetically modified (GM) soy and GM maize diet (Carman 2013, Journal of Organic Systems)

Claim supported by evidence?

The paper does not support the claim that GM crops cause stomach inflammation or increased uterus weight.

Summary

• The researchers pick out a few “statistically significant” results from a large number of tests
• Poor statistical methodology for assessing differences in inflammation
• The authors discounted an outlier before assessing difference in uterus weights, with inadequate justification

Study Conclusions

The data lend at best weak support for the conclusion that GM diet may lead to stomach inflammation and increased uterus size. The logistical and practical design of the experiment is strong with attention paid to blinding, ensuring the groups are comparable, and so on. However, it is let down by an inappropriate choice of statistical analysis methods as explained below.

Strengths/Limitations

• Randomized controlled study, in which many of the investigators and staff were blinded;  however it is not clear whether the statisticians were blinded.
• Data analysis methods are clearly set out in the methods
• A large number of statistical tests have been carried out, with no adjustment for multiplicity (see glossary), vastly increasing the potential for false positives
• An inappropriate comparison has been made with regards to stomach inflammation: “severe” inflammation has been assessed in isolation, rather than as part of the “nil/mild/moderate/severe” scale.
• The authors claim that the sample size is large enough “to be able to determine statistical significance for key toxicological outcomes” without giving any statistical justification for this statement.
• The clinical significance of the difference in uterus size is not made clear. 0.084 and 0.105 percent of body weight may well be within the normal range; this is not made clear. It is just assumed that a larger uterus size is worse (this is not my area of expertise however).

Glossary

Multiplicity – the problem of comparing two groups in terms of a large number of attributes. As the number of attributes increases, it becomes more likely that the two groups will appear to differ on at least one attribute purely by chance.

Any specific expertise relevant to studied paper (beyond statistical)?

No.