Exploratory or confirmatory?

In last week's issue of Science, outgoing editor Marcia McNutt was interviewed (Shell, 2016) on the occasion of beginning a term as President of the National Academy of Sciences.  I am going to reproduce a lengthy quote from the interview.

At Science, the paradigm is changing.  We're talking about asking authors, 'Is this hypothesis testing or exploratory?'  An exploratory study explores new questions rather than tests an existing hypothesis.  But scientists have felt that they had to disguise an exploratory study as hypothesis testing, and that is totally dishonest.  I have no problem with true exploratory science.  That is what I did most of my career.  But it is important that scientists call it as such and not try to pass it off as something else.  If the result is important and exciting, we want to publish exploratory studies, but at the same time make clear that they are generally statistically underpowered, and need to be reproduced.

Bravo, Dr. McNutt!  DTLR agrees completely with the sentiment here.  It matters because the statistical dressing that accompanies much scientific research is usually only appropriate for confirmatory studies, or those that McNutt calls hypothesis testing, rather than hypothesis finding (exploratory).  It is rare to find the editor of a major scientific journal express this view in such a crisp, precise manner.   DTLR hopes that her successor, and other editors and referees of scientific journals, follow the lead set by McNutt.  DTLR also recommends all readers of this blog to take a look at Tukey (1980).

Reference

Ellen Ruppel Shell, 2016:  Hurdling obstacles:  Meet Marcia McNutt, scientist, administrator, editor, and now National Academy of Sciences president.  Science, vol. 353, pp. 116-119.

John W. Tukey, 1980:  We need both exploratory and confirmatory.  The American Statistician, vol. 34, pp. 23-25.