Rush Holt is Out of Touch

A few weeks ago in Science, Dr. Rush Holt, CEO of the American Association for the Advancement of Science (AAAS) published an op-ed, “A tale of two cultures,” where he advocates science communication to “rebuild the public's understanding and appreciation” of “science and evidence-based thinking”. He continues, “It must be achieved by demonstrating trustworthiness and the extraordinary effectiveness of science in confronting questions and problems.” About a year ago, Dr. Holt testified before the House Committee on Science, Space, and Technology, that “I'm here to say don't try to reform the scientific process. It has served us well and will serve us well.”

DTLR finds the comments in bold above offensive. For many years on this blog, I have decried the perverse incentives for scientists that has led to sloppy methodology, publication bias, and dissemination of non-reproducible results. Scientific research in this country cannot continue under the status quo – reform is badly needed. Because so much academic and government science is driven by federal funding, it is the task of federal agencies and the Congress to participate in such a reform. Holt's testimony makes it sound like science is just fine the way it is, and please leave us alone to do it. This is not only anti-scientific but anti-democratic. The taxpayers fund science – they deserve for their money to produce reproducible results, not perpetuate a glass-bead game that rewards productivity, not reproducibility.  Non-reproducible research does not demonstrate "trustworthiness" and most certainly does not make science effective in "confronting questions and problems."

Rush Holt is out of touch with the crisis of non-reproducible research and self-destructive incentives built into the infrastructure of our profession.  He is a plasma physicist and former member of Congress from New Jersey, but evidently he is the wrong person to lead a major scientific society.

For more on non-reproducible research, a good summary may be found in NPR reporter Richard Harris' book, Rigor Mortis:  How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions, published last year by Basic Books.

The Physical Review journals landmark papers: fluid dynamics?

The Physical Review series of journals, published by the American Physical Society (APS), is celebrating its 125th anniversary by making a curated set of landmark papers available for free download here.  DTLR encourages readers to take a look.

The question I'd like to ask here is, how many of these milestone papers are on fluid dynamics?  I found only one, the 1986 paper by Frisch, Hasslacher, and Pomeau introducing lattice gas cellular automata (LGCA).  This is a computational method for simulating fluid flow described by the Navier-Stokes equations.

Of course, the papers by Onsager (1931, in 2 parts) on reciprocal relations in irreversible processes, has some relevance, for instance to mass diffusion.  Superfluids are represented by the observation of superfluidity in liquid helium-3 by Osheroff, Lee, and Richardson (1972), and the prediction of topological phase in 2D superfluids by Nelson and Kosterlitz (1977).  In condensed matter physics, the prediction of a hexatic phase, between solid and liquid, was predicted by Halperin and Nelson (1978) in their theory of 2D melting.  These are not really papers on fluid dynamics per se, but certainly related.

However, for the most part, the great achievements in classical fluid dynamics have not generally been published in the Physical Review series of journals.  In 1970 the Physical Review was split into four sections, with section A covering "General Physics" including fluid dynamics.  When Physical Review E was founded in 1993, fluid dynamics and plasma physics moved to this new journal, along with other topics such as statistical, nonlinear, and biological physics, and more recently soft matter.  In 2016 it was joined by a new journal, Physical Review Fluids, created due to the mass defection of the editorial board of Physics of Fluids due to disagreements with the publisher (the American Institute of Physics, AIP).  Of course, Physical Review Letters and Reviews of Modern Physics have occasionally included fluid dynamics papers.


However, for much of the post-World War II era, the two most elite journals in fluid dynamics research have been the Journal of Fluid Mechanics (founded in 1956, published by Cambridge University Press) and Physics of Fluids (founded in 1958), neither of which belong to the APS family of journals.  The various engineering societies also have journals that cover fluid dynamics, and a number of commercial publishers have journals in the field as well.  This is one reason why so few fluid dynamics papers show up in the APS list of milestone papers.  I've certainly read and enjoyed many fluid dynamics papers from the Physical Review family, but apparently none but Frisch et al. (1986) makes it onto the list of landmarks (which, the editors assure us, is so elite that several Nobel Prize-cited papers didn't make it either).

A physicist attends JMM

The Joint Mathematics Meetings (JMM) are advertised as the world's largest mathematics conference. This year's meeting was held in San Diego, CA, last week. DTLR attended despite his mild allergy to pure mathematics. This was my first time attending JMM, and I chose the sessions I attended with great care, and learned a lot. Some highlights are discussed below.

San Diego Convention Center, site of JMM 2018.

Physics

The best talk given by a mathematician was a physics talk, “Toy models,” by Stanford professor Tadashi Tokieda. He used a series of toys exhibiting unexpected, puzzling, and surprising behavior to illustrate ideas in physics, using almost no mathematics at all. Rather, he relied on qualitative reasoning and dimensional analysis rather than direct computation. After I returned home I discovered that many of his examples, and others, can be found in his Youtube videos, some of which are collected here. He was an exceedingly entertaining speaker – I was not bored for even one second. I would rate this as the best talk of the conference.

Computer scientist Dana Randall (Georgia Tech) gave a presentation on statistical physics, “Emergent phenomena in random structures and algorithms.” She discussed phase transitions in lattice gases, randomized algorithms, and swarm robotics, among other topics. (Phase transitions were also one of the topics addressed by Tokieda.)

Fluid Dynamics

Edriss S. Titi (Texas A&M and Weizmann Institute) provided a review of mathematical results on existence, uniqueness, and regularity of solutions for the incompressible Euler and Navier-Stokes equations under various conditions, such as 2D vs. 3D flows, and types of initial conditions. This is of course the subject of one of the million dollar prizes offered by the Clay Mathematics Institute. Rayleigh-Benard convection was given as an example. 

Isabelle Gallagher (University of Paris Diderot) presented a review of mathematical results connecting the Newton hard-sphere gas model to the Boltzmann transport equation from the kinetic theory of gases, and to the Navier-Stokes equations for a continuum fluid. One of the puzzles is how does a fundamentally reversible system – Newton's laws applied to a gas of hard spheres – result in irreversible behavior characterized by the second law of thermodynamics (reflected in both the Boltzmann and Navier-Stokes models). Her answer to this is the Ehrenfest experiment, where such a gas begins in one chamber, and at a certain time the portal to a second chamber is opened. Eventually an equilibrium is reached where both chambers have approximately the same number of particles. The key is the number of particles. If there are just two particles, nothing remarkable is observed. However, when there are many particles, the most likely states of the system are those near the equilibrium state. Thus, the statistical properties of a completely deterministic system are consistent with the second law of thermodynamics. We might think of this as an emergent behavior, not unlike those discussed in Dana Randall's talk.

Data Analysis

Topologist Gunnar Carlson (Stanford) discussed “Topological Modeling of Complex Data”. Here the “model” is not a statistical model, but rather a network model that attempts to capture the shape of data. The idea is to apply overlapping bins to the data along some of the predictor variables, and cluster the data along these axes into nodes. Nodes with overlapping data are connected, forming a network graph.

Applied mathematician Tamara G. Kolda (Sandia National Labs) spoke about tensor decompositions, particularly a decomposition known as CP (canonical polyadic). These tensor decompositions are not  orthogonal, but the idea is to essentially project high dimensional data onto what I will call basis tensors. Randomization plays a key role in the algorithm.

Neither of these presentations is by a statistician, and neither addresses statistical inference, rightly so in my view. Rather, they belong to exploratory data analysis (EDA), a field that Carlson reminds us was invented by (topologist) John Tukey.

Computer Science

Harvard computer scientist Cynthia Dwork (joint appointment with Microsoft) presented a talk on differential privacy. Algorithms that exhibit differential privacy are randomized algorithms that respond to a query to a data base, and provide the following guarantee. A specific person's decision to be included (or not) in the data base should not affect the outputs of the algorithm. Let two data sets differ by a single individual's record (she is present in one data set, and a different randomly sampled person from the population replaces her in the second). Any outcome output from the algorithm run on either data set will be almost equally likely. Algorithms with the differential privacy property are also inherently robust in the statistical sense, and can be used for adaptive/exploratory reuse of the data for statistical modeling.

A theme is evident: the talks by Randall, Kolda, and Dwork all involve randomized algorithms in computer science, not a topic that I thought about when I was a student 20 or so years ago.

What about Math?

I did attempt to attend some actual pure math talks: Alissa Crans (Loyola Marymount University) on “Quintessential quandle queries” and Craig Huneke (Virginia) on “How complicated are polynomials in many variables”. I had not heard of quandles before; the speaker related them to both groups and knots. An application to cryptography was mentioned but not dwelled on. She did at least bring a prop (a giant tetrahedron) that she used for illustration. No applications were mentioned at all in the polynomial talk, which focused on something called Stillman's conjecture. I also attended a talk by a distinguished historian of mathematics, Joseph Dauben (CUNY), on the history of Chinese math (actually, the history of Chinese historians of mathematics).

A slide from Alissa Crans' lecture on quandles, showing the definition and the mathematician who coined the term.

I also attended a panel session on careers in business, industry, and government, which probably could have run longer than it did. I was pleased to see a high level of interest in this session, but sad that many math students aren't sure how or even whether to pursue such careers. It's a good thing that they came to this session, but their home departments should be doing more to stimulate interest and provide practical resources for such career development. There was also a talk by applied mathematician Stephen Hobbs (Space and Naval Warfare Systems Center) on a simple model for deploying aircraft-carrier based resources in a humanitarian aid scenario. Finally I attended a few sessions on statistical education, including one on developing a data science program within a mathematics department.

The exhibit hall was a delight, with many book and software vendors offering their wares.  The National Security Agency had a recruiting booth which featured an original Enigma encryption device, which attendees were invited to interact with (unlike museum pieces that remain behind glass).

An original ENIGMA device at the NSA booth.

Of course, San Diego is a nice spot to go in the winter, with many extracurricular delights.

Beef empanadas at a Spanish restaurant, Cafe Sevilla, in the Gaslamp District of San Diego.

Videotaping experiments?

Today's issue of Nature has an interesting commentary by Timothy D. Clark, advocating videotaping of experiments.  His motivation is to combat scientific fraud.  However, there are broader reasons to consider the idea.  Clark alludes to some, and here is another.  Back when Google Glass came out, I read about a scientist who recorded the execution of her lab protocol simply as a means of documenting what was done.  This can be an extremely valuable supplement to a written protocol, as a video recording is more likely to capture "folk knowledge" within a lab, that nobody thinks to write down.  In other words, such practices could enhance reproducibility.

Of course, videotaping experiments is a long tradition in fluid dynamics, dating back to Henri Benard's films of what we now call von Karman vortex streets, in the first decades of the 20th century.  Today, image analysis methods are often used to extract data from moving images of fluids.

The other point made by Clark's post is that the burden of proof for allegations of misconduct is on the accuser, rather than the accused.  He makes a point about the trust-based nature of the scientific enterprise.  Shifting the burden of proof to the authors not only reduces the likelihood of fraud, but enhances the likelihood of reproducibility.    DTLR endorses Clark's proposal.

A less dismal science, redux

Last summer DTLR made a rare foray into the behavioral sciences, commenting on a critique of academic economics found in The Economist. Well Bloomberg View has an editorial this week with a more comprehensive critique of academic economics, one that resonates well with DTLR.  Check it out.

Happy New Year

DTLR expects the new year to bring more scrutiny to non-reproducible research, and poor study design, conduct, analysis, and reporting.  Here is an example from last month in Nature.

"To err is human, but so often?" - David Freedman

Nature's editorial this week discusses the unleashing of the "statcheck" computer program on psychology journal articles.  Evidently it is an automated mechanism to detect errors in the calculation of p-values reported in published papers.

While I do not object to anything they said in the editorial, my concern is that there is very little penalty for carelessness in scientific research.  P-values are actually the least of my concerns; of greater concern are errors or even sub-optimal practices in the design, execution, and reporting of research.  Statistical inference is of no value if these other issues are present, and even when not present, statistical inference remains of incredibly limited value compared to a descriptive presentation of the data.  There are several reasons for this, such as:

• Statistical inference presumes some kind of generalization, usually to a larger, stable population of which the data in the study can be thought of as representative.  This is rarely justified.
• The statistical analysis adds information to the data  in the form of an assumed probability model.  This model's assumptions may well influence the outcome more than the data does.
• Statistical inference is an inherently confirmatory activity, while most research is exploratory.  Statistical models in this context are overfitted to the data, and the generalization implied by statistical inference is invalid.

Nonetheless, sloppy calculation is a sign of carelessness, and for this reason the "statcheck" episode has certainly done a service, if it dis-incentivizes future carelessness.  On the other hand, legitimate criticisms of "statcheck's" own error rate have been raised.  I see this as the needed back-and-forth in the ongoing discussion of reproducible research, and accusations of "harassment" on the part of "statcheck's" creators are over-sensitive and unwarranted.