Responses to "When Mice Mislead"

This past week's issue of Science (the Jan. 24, 2014 issue) has two letters to the editor, responding to a report last November, "When Mice Mislead" by Jennifer Couzin-Frankel, which I discussed in an earlier post.  The first letter, by Richard Traystman and Paco Herson, points to earlier findings, similar to those reported by Couzin-Frankel, in the stroke research community.  Most importantly, they assert that "It is unlikely that poor methods used in animal studies account for all the negative clincial trials that have been performed based on preclinical studies.  After all, some investigators do perform appropriate experiments, and even those studies rarely lead to positive clinical trials."  The authors point to the fact that mouse studies are usually done with healthy young mice, whereas human subjects in neuroprotective drug clinical trials are often older and have many co-morbidities.  They propose that aged mice with comorbid diseases be used in stroke trials, as a better animal model of human disease.

The second letter is from statistician Gary Churchill.  He zeroes in on one key question:  "Was the result replicated in more than one genetic background?"  He goes on to identify two "root causes" for nonreproducible research:

Science today is driven by an incentive system that often rewards precedence and impact over quality of the work.  Statistical training of scientists often emphasizes analytical techniques over experimental design and quantitative reasoning.  These are systemic problems that will not change without substantial effort

Meanwhile, Churchill endorses the message of Couzin-Frankel's article with his maxim:  "Be wise, randomize."

I think that both of these letters add value to the original piece by Couzin-Frankel. In particular, Churchill's second "root cause" is particularly interesting, as both statisticians and lay scientists or mathematicians who teach statistics are all guilty of overemphasizing methodology, modeling, and inference at the expense of study design and critical thinking. 

References

Jennifer Couzin-Frankel, 2013: When mice mislead. Science, 342: 922-925.

Richard J. Traystman and Paco S. Herson, 2014:  Misleading results:  translational challenges.  Science, 343:  369-370.

Gary Churchill, 2014:  Misleading results:  don't blame the mice.  Science, 343, 370.