The AMS statement on weather analysis and forecasting

Following up on my last post on meteorology, I want to draw readers' attention to the American Meteorological Society's information statement on weather analysis and forecasting, which appeared in the Society's Bulletin recently, and can be found online here.  I found it to be very informative.

Dr. Marshall Shepherd's weather pet peeves

Check out Dr. Marshall Shepherd's blog post, "10 Common Myths and Misconceptions about the Science of Weather" at Forbes.com.  He is a distinguished professor at the University of Georgia and a former president of the American Meteorological Society.

Self-correction and an open research culture in science

Back in the June 26 issue of Science, there was a pair of commentaries by Alberts et al. (2015) and Nosek et al. (2015) entitled, respectively, "Self-correction in science at work" and "Promoting an open research culture."  Both articles are reactions to concerns about non-reproducible research.  The first one focuses on incentives, investigations of misconduct, and ethics education.  The second article deals with transparency, and introduces a set of guidelines called TOP (Transparency and Openness Promotion).  It discusses various standards for transparency, with different levels.  While I do not think this pair of articles captures the whole scope of the irreproducibility crisis, they do provide much food for thought on how we should address it.  Thus I recommend both articles to DTLR readers.

One passage in Nosek, et al. (2015), discussing the preregistration of studies and analysis plans, caught my eye.  "Preregistration of analysis plans certify the distinction between confirmatory and exploratory research, or what is also called hypothesis testing versus hypothesis generating research.  Making transparent the distinction between confirmatory and exploratory methods can enhance reproducibility."  I fully endorse this perspective.  I have met scientists who fail to understand the point made here so succinctly, and it really deserves emphasis.

References

B. Alberts, et al., 2015:  Self-correction in science at work.   Science, 348:  1420-1422.

B. A. Nosek, et al., 2015:  Promoting an open research culture.  Science, 348:  1422-1425.

Some nice physics in the popular press

I was delighted to see a couple of physics items in the popular press.  Last week the New York Times had this opinion piece by physicists Adam Frank and Marcelo Gleiser, "A crisis at the edge of physics."  There isn't much new here for DTLR readers, as last year we've covered similar ground in our discussion of Jim Baggott's book, Farewell to Reality.  Helen Quinn's term "scientific metaphysics" (see my previous post) might be a good one to use to describe what Baggott less charitably calls "fairy tale physics".  Nonetheless I take a dim view of efforts to de-emphasize the importance of empirical testing.  Such efforts smack of wishful thinking, in my view.

Then Forbes had a piece by physicist Chad Orzel defending his choice to work in atomic and molecular physics.  Titled "Particle and astro aren't the only kinds of physics", it is absolutely delightful, except for the swipe at biomedical researchers near the end!  Of course, condensed matter physics is the largest sub-field of physics, but it does seem true that particle physics, astrophysics, and cosmology are the branches of physics that get the most traction in the popular media.  Physics needs more evangelists like Orzel to call attention to the many and varied other subfields of physics.

Science is done in an artificial environment

I would like to recall a Physics Today "Reference Frame" column from a number of years ago by distinguished particle physicist, Helen Quinn (2009), "What is Science?"  This is a particularly philosophical piece, which among other things introduces the term "scientific metaphysics" to deal with extrapolations of scientific theory and speculation into regimes that are not, in principle, empirically testable, such as the" many worlds" interpretation of quantum theory.

However, another part of the article calls my attention today.  I would like to quote an entire paragraph:

Science is done in an artificial environment, where its logic can develop without a need for immediate action.  That unnatural environment allows science to yield powerful and unexpected new options for eventual action.  It is important to note, however, that some applications of science, such as medicine, cannot wait until all questions are resolved.  Medical practice can be based on the best available scientific knowledge and theory, but it must often apply them in untested regimes.  Much of the public's feeling that science is always changing its conclusions comes from changes in medical advice that occur when new scientific knowledge overrides the previously best guesses of medical practice.

I've long agreed that "Science is done in an artificial environment," and I have gravitated towards applied science, partly as a result of discomfort with the ivory tower nature of basic research.  The aspect of science that Quinn describes above is one that has not been emphasized much by others, in my experience.  However, I think it is important for both scientists and non-scientists to understand this point, and Quinn puts it more eloquently than I could have.

I would only disagree with the gist of the final sentence.  In medicine, much of the flip-flopping conclusions that Quinn speaks of are due to the prevalence of studies based on observational, not experimental, data.  Lacking any attribution to causality, such studies really do give the impression of changing conclusions, which is detrimental to both scientists and non-scientists alike.

Reference

Helen Quinn, 2009:  What is Science?  Physics Today, 62 (7):  8-9.

The grubby details of reproducing research

This week in Nature, Richard van Noorden writes of some preliminary findings of the Reproducibility Initiative:  Cancer Biology, presented at a conference in Brazil.  It is an interesting report, though I feel somewhat uncomfortable about the meta-analysis type statistical significance measure that they plan to calculate.  Nonetheless, the effort seems a worthy one, and DTLR awaits the final results.

More pieces of the nonreproducibility puzzle

A couple of recent news features have shed light on pieces of the reproducible research puzzle. Back in February, Jill Neimark in Science wrote about contaminated cell lines.  And just this week, Monya Baker in Nature wrote about batch-to-batch variation and non-specificity of antibodies.  Cells and antibodies are workhorses of modern biological research, and growing attention is needed to these potential sources of error.  I commend both of these articles to DTLR readers.

References

M. Baker, 2015:  Blame it on the antibodies.  Nature, 521:  274-276.

J. Neimark, 2015:  Line of  attack.  Science, 347:  938-940.