Nature keeps the heat up on reproducible research

This week's issue of Nature has a good article by Monya Baker on a wide-ranging survey of scientists about reproducible research, and a related editorial.  DTLR is most encouraged by the final table in Baker's article, the ratings of factors most likely to improve reproducibility.  "More robust experimental design" received the most combined "likely" and "very likely" ratings.  I think that this is the right answer.  Also highly ranked were "better mentoring/supervision" and "better understanding of statistics".  This latter one is a tough call, as statisticians themselves seem not to have reached a consensus on how to move forward, as evidenced in the extensive Discussion items published along with the American Statistical Association's Statement on Statistical Significance and P-values, posted in early March.

DTLR expresses thanks to Nature for keeping the drums beating on reproducible research.  The issue is very visible right now, and the community should strike while the iron is hot, in terms of reforming the infrastructure of our community (laboratory practices, publication standards, and incentives for grant funding, promotion, and tenure).  Mis-aligned incentives are ultimately the cause of non-reproducibility, though methodological issues (poor study design and execution, inappropriate use of statistical methods, etc.) are key enablers.

The water watchdog

DTLR supports the views of Prof. Marc Edwards, expressed in interviews with the Chronicle of Higher Education (with Steve Kolowich, here) and Science Magazine's Working Life (with Rachel Bernstein, here), regarding the mis-aligned incentives for academic scientists, and other topics.  He is one of the experts worked to "uncover and address the elevated lead levels in drinking water in Flint, Michigan" (as Bernstein wrote).

Gravitational waves and colliding black holes

DTLR has been dormant for nearly a half-year.  However a significant discovery has been reported recently that is worthy of note.  The LIGO (Laser Interferometry Gravitational-Wave Observatory) detected the emission of gravitational radiation from a source event that appears to be the collision of two black holes.  The event marks the dawn of the age of gravitational wave astrophysics.  We are again grateful to be witnesses to the making of physics history.  Congratulations to the LIGO and VIRGO collaborations.

Reference:

B.P. Abbott, et al., 2016:  Observation of gravitational waves from a binary black hole merger.  Physical Review Letters, 116:  061102.

The role of institutions in promoting reproducible research

Nature published a superb commentary yesterday by Glenn Begley, Alistair Buchan, and Ulrich Dirnagl, advocating reforms among institutions to help reduce irreproducible research.  I have little to add except unabashed praise.  DTLR endorses the views expressed in the commentary.

The AMS statement on weather analysis and forecasting

Following up on my last post on meteorology, I want to draw readers' attention to the American Meteorological Society's information statement on weather analysis and forecasting, which appeared in the Society's Bulletin recently, and can be found online here.  I found it to be very informative.

Dr. Marshall Shepherd's weather pet peeves

Check out Dr. Marshall Shepherd's blog post, "10 Common Myths and Misconceptions about the Science of Weather" at Forbes.com.  He is a distinguished professor at the University of Georgia and a former president of the American Meteorological Society.

Self-correction and an open research culture in science

Back in the June 26 issue of Science, there was a pair of commentaries by Alberts et al. (2015) and Nosek et al. (2015) entitled, respectively, "Self-correction in science at work" and "Promoting an open research culture."  Both articles are reactions to concerns about non-reproducible research.  The first one focuses on incentives, investigations of misconduct, and ethics education.  The second article deals with transparency, and introduces a set of guidelines called TOP (Transparency and Openness Promotion).  It discusses various standards for transparency, with different levels.  While I do not think this pair of articles captures the whole scope of the irreproducibility crisis, they do provide much food for thought on how we should address it.  Thus I recommend both articles to DTLR readers.

One passage in Nosek, et al. (2015), discussing the preregistration of studies and analysis plans, caught my eye.  "Preregistration of analysis plans certify the distinction between confirmatory and exploratory research, or what is also called hypothesis testing versus hypothesis generating research.  Making transparent the distinction between confirmatory and exploratory methods can enhance reproducibility."  I fully endorse this perspective.  I have met scientists who fail to understand the point made here so succinctly, and it really deserves emphasis.

References

B. Alberts, et al., 2015:  Self-correction in science at work.   Science, 348:  1420-1422.

B. A. Nosek, et al., 2015:  Promoting an open research culture.  Science, 348:  1422-1425.