In praise of high journal acceptance rates?

A Penn State professor of astronomy and astrophysics, Jason Wright, published a commentary in the February 2020 issue of Physics Today, titled "High journal acceptance rates are good for science".  He notes that the journals he publishes in have an 85% acceptance rate.  He says that "Therefore nearly all significant astronomical results submitted to those journals that are not obviously fatally flawed are likely to be published....it is the sign of a healthy culture of science, and astronomy is better for it."  He writes about how this state of affairs helps to avoid publication bias, because paper rejection is often influenced by reasons other than quality:  "scientific taste, politics, professional advantage, and science's inherent conservatism."  He even writes that "it's important that scientists be allowed to be wrong in the literature, as long as they have made no errors."  He concludes that "referees best serve when they act not as gatekeepers but as editorial consultants and independent voices that offer construcive criticism that improves submitted papers."

DTLR finds much merit in Wright's arguments.  However, I do think referees and editors have to be gatekeepers when it comes to one point that Wright alludes to, but does not emphasize.  I and many others have seen methodologically unsound research published, even in prestigious journals with low acceptance rates, in the life and social sciences.  A methdological critic would have been able to reject the paper before seeing any of the data.  This is a collective failing of authors, referees, and editors, who are often themselves untutored in even basic principles of research design and execution. There has been much discussion of these phenomena, for instance, in a special issue of the Lancet in 2014, and in closely related discussions of reproducible research (for instance).  Sadly, that this has continued to be a recognized problem for almost 15 years is a sign of an unhealthy culture of science.

One proposed remedy has much appeal:  "Results-Blind Manuscript Evaluation" (RBME), initially proposed by Joseph Locascio over 20 years ago (eg, Locascio, 2019).  This involves a two-stage manuscript review, where a mansucript is first evaluated on the basis of the Introduction and Methods sections, without knowledge of the results.  Methodologically unsound research can be rejected out of hand at this stage; if not, the manuscript moves to the second stage where its entirety is reviewed, though the decision to accept or reject may still not be based on the results, but only on the soundness of execution, analysis, and presentation thereof.  See the cited paper by Locascio for details.  RBME would not solve all the problems, but would go a long way in changing the incentives.

DTLR believes that a scientific journal should combine the insights of Wright and Locascio in its efforts to fight publication bias, while ensuring that research with a chance of contributing to, rather than misleading, the work of others, sees the light of day.

References

J. J. Locascio, 2019:  The impact of results blind science publishing on statistical consultation and collaboration.  The American Statistician, 73 sup 1:  346-351.

J. Wright, 2020:  High journal acceptance rates are good for science.  Physics Today, 73 (2):  10-11.

The Women of Fluid Mechanics: Personal Stories and Practical Advice

Last month's issue of APS News featured an article by Otani & Hu, titled "The Women of Fluid Mechanics:  Personal Stories and Practical Advice".  The article was a commentary based on a panel session at last year's annual APS Division of Fluid Dynamics (APS-DFD) meeting, which featured four female professionals in the field (3 faculty members, and the fourth the founder of a very popular blog and miniature social media empire, FYFD).  (Disclosure:  I am a "patron" of FYFD through the Patreon website.)

Among other things, the authors write that "In spite of their intellect, work ethic, and accomplishments, all of these women have faced and witnessed disrespect and scrutiny based on their gender."  It is indeed disappointing that this still happens at the end of the second decade of the 21st century.   I myself witnessed inappropriate behavior by male physics faculty in the late 1990s, as I wrote about here, though that episode was not related to fluid mechanics. 

DTLR supports efforts to reduce the barriers for women in fluid mechanics research.  One female fluid dynamicst (whom I have never met) certainly has had a key influence, through her published work, on my brief years of research in fluid mechanics.  She is cited in every fluid dynamics paper I have co-authored, and her name appears in the abstract of 3 of the 4 refereed papers of mine in the field.  Hopefully more women's names will appear in the reference lists of fluid dynamics research papers as time goes on.

Reference

Courtney Otani and David Hu, 2020:  The women of fluid mechanics:  personal stories and practical advice.  APS News, 29 (3):  8.

Feynman's 1964 Messenger Lectures

Happy New Year.  DTLR would like to point its readers to Richard Feynman's 1964 Messenger Lectures at Cornell University, which Bill Gates and Microsoft have made available for viewing at Project Tuva.  These lectures also appeared in book form (Feynman, 1967).  DTLR highly recommends this lecture series.

Reference

Richard Feynman, 1967:  The Character of Physical Law.  MIT Press.

Machine learning and fluid mechanics

Recently Physical Review Fluids published an invited op-ed, "Perspective on machine learning for advancing fluid mechanics" by Brenner, Eldredge, and Freund.  As one who has dabbled in both fields, I found their comments illuminating and balanced, conveying both astonishment and excitement at the achievements and potential of machine learning methodology, as well as its potential pitfalls and limitations.  I share their astonishment that in some cases, deep neural networks have been able to generalize beyond the training data.  This remains an ill-understood phenomenon.  We do not know under what circumstances such generalization can reliably occur, and I believe any such claims about these generalizations must be validated with independent data sets.  Regardless, however, I am in broad agreement with the authors' views, including their conclusion that machine learning methods have potential for high impact "so long as outcomes are held to the long-standing critical standards that should guide studies of flow physics."

Reference

M. P. Brenner, J. D. Eldredge, and J. B. Freund, 2019:  Perspective on machine learning for advancing fluid mecanics.  Physical Review Fluids, 4:  100501 (7 pages).

Science Magazine embraces replication (and gets its nose bloodied)

Science has put its money where its mouth is.  In this week's issue, the editor Jeremy Berg writes about "Replication Challenges".  They have published not one, but three NIH-funded replication studies of an earlier paper.  That is the good news.  The bad news is that a sister journal published a 10-patient human clinical trial, also based on the original paper, in parallel with the 3 replication studies.  Like all 3 replication studies, the human study was not able to reproduce the effect claimed in the original paper.

Can you see why journals may be reluctant to press forward with publishing replication studies?  They can certainly get their noses bloodied.  DTLR commends the editor of Science for restating his commitment to publishing replication studies.

Jerry Gollub, 1944-2019

The first substantive post I wrote for this blog, back in 2013, was a commentary on a paper of Jerry Gollub's, on continuum mechanics in physics education.  I learned recently that sadly, Dr. Gollub lost his life earlier this summer.  An obituary by his renowned collaborator, Harry Swinney, has been posted at Physics Today.  See also this obituary posted at his institution, Haverford College.  I never met or corresponded with Dr. Gollub, but his work has been an influence on mine since I was a junior in college, when I studied from his book with Gregory L. Baker, Chaotic Dynamics:  An Introduction.  In graduate school, the book he edited with Swinney, Hydrodynamic Instabilities and the Transition to Turbulence, was of equal importance to my work.  In my student days, I had these books checked out "long term" from the library, and eventually acquired copies of my own.  DTLR joins the physics and fluid mechanics communities in mourning the loss of this inspiring scientist. 

Since I completed my doctoral thesis in 2001, several other titans I cited in it have passed.  These include V. I. Arnol'd (1937-2010), Robert P. Behringer (1948-2018), Philip G. Drazin (1934-2002), Louis N. Howard (1929-2015), Daniel D. Joseph (1929-2011), Leo P. Kadanoff (1937-2015), E. Lothar Koschmieder (1929-2017), Robert Kraichnan (1928-2008), Olga Ladyzhenskaya (1922-2004), Edward N. Lorenz (1917-2008), John L. Lumley (1930-2015), Steven R. Orszag (1943-2011), William Hill Reid (1926-2016), Philip G. Saffman (1931-2008), James B. Serrin (1926-2012), and Norman J. Zabusky (1929-2018).  There may certainly be others I am not aware of.

References

G. L. Baker and  J. P. Gollub, 1996:  Chaotic Dynamics:  An Introduction.  Second edition.  New York:  Cambridge University Press.  (First edition, 1990.)

H. L. Swinney and J. P. Gollub (eds.), 1985:  Hydrodynamic Instabilities and the Transition to Turbulence.  Second edition.  Topics in Applied Physics, Vol. 45.  Berlin:  Springer.  (First edition, 1981.)

The New Yorker weighs in on statistics

Hannah Fry writes in the New Yorker this month about the benefits and pitfalls of statistics.  Highly recommended.