The solar corona

We've been hearing a lot about "corona" lately, but a remarkable photo of the solar corona has been making the rounds, courtesy of the European Space Agency's Solar Orbiter satellite mission.  (The link is to Elizabeth Gibney's Nature article about it.)  The photo is an excellent complement to a high-resolution photo of solar granulation released earlier this year by the ground-based Daniel K. Inouye Solar Telescope in Hawaii (see Alexandra Witze's Nature article here.)  Along with NASA's Parker Solar Probe, and a new European ground-based telescope planned for the Canary Islands, we may be entering an exciting period to do solar physics.

Escape From Model-Land

Earlier this week, Mark Buchanan's column in Nature Physics featured "The Limits of a Model" , concerning  the pitfalls of epidemiological forecasting, of course a topic very much in the news in the last few months.  Near the end Buchanan cites an intriguing paper by Thompson & Smith (2019), "Escape From Model-Land", a bracing discussion of why one should be very humble about using mathematical/empirical models.  It calls to mind another entertaining piece from over a decade ago, the "Financial Modeler's Manifesto" by Emanuel Derman and Paul Wilmott.

DTLR strongly recommends the Thompson & Smith piece, a candidate for required reading by every applied mathematician, applied statistician, and mathematical/computational modeler in every discipline.

Reference

Erica L. Thompson and Leonard A. Smith, 2019:  Escape From Model-Land.  Economics:  The Open-Access, Open-Assessment E-Journal, 13:  2019-40.

A critique of "AI Feynman"

Udrescu & Tegmark (2020) describe a neural network-based machine learning algorithm for "symbolic regression", which they define as determining the form of a functional relationship between a response variable and a list of input variables from a data set.  Kepler's realization that Mars' orbit is elliptical, inferred from observational astronomy data, is the prime example they give.  Their "AI Feynman" algorithm has a number of modules, including automated dimensional analysis, polynomial fitting, neural network-based data interpolation, tests for symmetry and separability, and so on.  The paper concludes with the ambitious declaration, "We look forward to the day when, for the first time in the history of physics, a computer, just like Kepler, discovers a useful and hitherto unknown physics formula through symbolic regression!"  While the paper reports important accomplishments and leaves the door open to further improvements in the authors' methodology, DTLR believes that the effort is less than meets the eye.  Here I will discuss some methodological weaknesses, most of which the authors acknowledge, as well as a conceptual weakness.

Methodological weaknesses

The authors compare their "AI Feynman" algorithm to a commercial software called Eureqa.  They defined a set of 100 algebraic equations from the Feynman Lectures and compared the two algorithms' ability to discover the correct functional relationship, using simulated data from the "true" relationship (without and with Gaussian noise in the response variable, but not in any of the independent variables).  Data for the input variables was sampled uniformly between 1 and 5, a rather arbitrary choice.  The authors triumphantly report that AI Feynman had a 100% success rate, while Eureqa achieved only 71%.  However, buried in the paper is the fact that they used the Feynman Lectures as a training set, on which their algorithm was tuned (ie, potentially overfit to).  Thus the comparison with Eureqa on this data set is wholly unfair, and the claimed 100% success rate is not generalizable.  Recognizing this, they define a set of 20 equations drawn from other classic physics texts, reporting that AI Feynman solved 90% while Eureqa solved only 15%.  Finally a third test set of somewhat arbitrary mathematical relationships was defined; AI Feynman achieved 67% success compared to Eureqa's 49%.  The authors explain that this last set of equations may not have the nice properties of physical laws that AI Feynman was designed to exploit.

Data from real experiments would rarely resemble the simulated data that the authors used.  At the early stages of an investigation, part of the domain of the input variables may even be experimentally/observationally inaccessible.  Thus the authors' comparison exercise seems artificial, though a necessary preliminary step before tackling real data.  Nonetheless it is important to point this out to readers who might find the reported performance metrics impressive.  The authors acknowledge that they did not include differential or integral equations in their evaluation, nor noise in the input variables, nor real data sets, though they hope to address all these weaknesses in future work.

A word about dimensional analysis:  In the authors' Newtonian gravity example (their Fig. 2), why were the temperatures of the masses not considered as candidate input variables?  Why is gravity completely ignored in a dimensional analysis of the ideal gas law (Lemons, 2017)?  In both cases, physical intuition is the answer.  The first step in any dimensional analysis is of course the selection of candidate variables to include in the functional relationship.  In the authors' exercise, the candidate variables were pre-selected from knowledge of the true relationship.  In reality, physical intuition (which requires training; it is not the same as natural intuition) must be used.  Santiago (2019) states that this step is the most difficult and requires the most experience.  The physicist generating the experimental data has often already made the decision on which variables should be measured, so the point may be moot, at least for physics problems.  However, if the method is extended to other, more empirical domains, such as public health, the social and behavioral sciences, and finance, it will run into trouble.  It is often far less obvious which variables should be considered even as candidates for inclusion, and dimensional analysis may cease to be an operative method.  In fact, it is often the case that you have little or no data on the input variables you really need, while you have plenty of data on variables of limited relevance to your problem. 

Conceptual weakness

When Max Planck first obtained his blackbody radiation law, as an empirical formula that satisfied constraints imposed both by the known data and physical intuition, he did not publish it.  Why?  Because at that moment, the formula was strictly empirical; it did not provide any physical insight.  Planck published it only after he developed a theoretical model of resonators in thermal equilibrium with the radiation, from which the formula could be derived.  A feature of his model was the quantization of energy.  The latter was not taken seriously until Einstein showed that energy quantization could also resolve the mystery of the photoelectric effect, superfically a completely different physics problem.  The new physics wasn't just one new equation, it was a new concept.  If energy quantization were an intrinsic property of nature, it would manifest in many physics problems, and physicists gradually discovered that this was indeed the case.  Symbolic regression is at best a contributing factor to discovering new physics; it cannot be the sole tool for doing so.

References

D. S. Lemons, 2017:  A Student's Guide to Dimensional Analysis.  Cambridge University Press, Sec. 1.8.

J. G. Santiago, 2019:  A First Course in Dimensional Analysis.  MIT Press, Sec. 5.2.

S.-M. Udrescu and M. Tegmark, 2020:  AI Feynman:  a physics-inspired method for symbolic regression.  Science Advances, 6:  eaay2631.

In praise of high journal acceptance rates?

A Penn State professor of astronomy and astrophysics, Jason Wright, published a commentary in the February 2020 issue of Physics Today, titled "High journal acceptance rates are good for science".  He notes that the journals he publishes in have an 85% acceptance rate.  He says that "Therefore nearly all significant astronomical results submitted to those journals that are not obviously fatally flawed are likely to be published....it is the sign of a healthy culture of science, and astronomy is better for it."  He writes about how this state of affairs helps to avoid publication bias, because paper rejection is often influenced by reasons other than quality:  "scientific taste, politics, professional advantage, and science's inherent conservatism."  He even writes that "it's important that scientists be allowed to be wrong in the literature, as long as they have made no errors."  He concludes that "referees best serve when they act not as gatekeepers but as editorial consultants and independent voices that offer construcive criticism that improves submitted papers."

DTLR finds much merit in Wright's arguments.  However, I do think referees and editors have to be gatekeepers when it comes to one point that Wright alludes to, but does not emphasize.  I and many others have seen methodologically unsound research published, even in prestigious journals with low acceptance rates, in the life and social sciences.  A methdological critic would have been able to reject the paper before seeing any of the data.  This is a collective failing of authors, referees, and editors, who are often themselves untutored in even basic principles of research design and execution. There has been much discussion of these phenomena, for instance, in a special issue of the Lancet in 2014, and in closely related discussions of reproducible research (for instance).  Sadly, that this has continued to be a recognized problem for almost 15 years is a sign of an unhealthy culture of science.

One proposed remedy has much appeal:  "Results-Blind Manuscript Evaluation" (RBME), initially proposed by Joseph Locascio over 20 years ago (eg, Locascio, 2019).  This involves a two-stage manuscript review, where a mansucript is first evaluated on the basis of the Introduction and Methods sections, without knowledge of the results.  Methodologically unsound research can be rejected out of hand at this stage; if not, the manuscript moves to the second stage where its entirety is reviewed, though the decision to accept or reject may still not be based on the results, but only on the soundness of execution, analysis, and presentation thereof.  See the cited paper by Locascio for details.  RBME would not solve all the problems, but would go a long way in changing the incentives.

DTLR believes that a scientific journal should combine the insights of Wright and Locascio in its efforts to fight publication bias, while ensuring that research with a chance of contributing to, rather than misleading, the work of others, sees the light of day.

References

J. J. Locascio, 2019:  The impact of results blind science publishing on statistical consultation and collaboration.  The American Statistician, 73 sup 1:  346-351.

J. Wright, 2020:  High journal acceptance rates are good for science.  Physics Today, 73 (2):  10-11.

The Women of Fluid Mechanics: Personal Stories and Practical Advice

Last month's issue of APS News featured an article by Otani & Hu, titled "The Women of Fluid Mechanics:  Personal Stories and Practical Advice".  The article was a commentary based on a panel session at last year's annual APS Division of Fluid Dynamics (APS-DFD) meeting, which featured four female professionals in the field (3 faculty members, and the fourth the founder of a very popular blog and miniature social media empire, FYFD).  (Disclosure:  I am a "patron" of FYFD through the Patreon website.)

Among other things, the authors write that "In spite of their intellect, work ethic, and accomplishments, all of these women have faced and witnessed disrespect and scrutiny based on their gender."  It is indeed disappointing that this still happens at the end of the second decade of the 21st century.   I myself witnessed inappropriate behavior by male physics faculty in the late 1990s, as I wrote about here, though that episode was not related to fluid mechanics. 

DTLR supports efforts to reduce the barriers for women in fluid mechanics research.  One female fluid dynamicst (whom I have never met) certainly has had a key influence, through her published work, on my brief years of research in fluid mechanics.  She is cited in every fluid dynamics paper I have co-authored, and her name appears in the abstract of 3 of the 4 refereed papers of mine in the field.  Hopefully more women's names will appear in the reference lists of fluid dynamics research papers as time goes on.

Reference

Courtney Otani and David Hu, 2020:  The women of fluid mechanics:  personal stories and practical advice.  APS News, 29 (3):  8.

Feynman's 1964 Messenger Lectures

Happy New Year.  DTLR would like to point its readers to Richard Feynman's 1964 Messenger Lectures at Cornell University, which Bill Gates and Microsoft have made available for viewing at Project Tuva.  These lectures also appeared in book form (Feynman, 1967).  DTLR highly recommends this lecture series.

Reference

Richard Feynman, 1967:  The Character of Physical Law.  MIT Press.

Machine learning and fluid mechanics

Recently Physical Review Fluids published an invited op-ed, "Perspective on machine learning for advancing fluid mechanics" by Brenner, Eldredge, and Freund.  As one who has dabbled in both fields, I found their comments illuminating and balanced, conveying both astonishment and excitement at the achievements and potential of machine learning methodology, as well as its potential pitfalls and limitations.  I share their astonishment that in some cases, deep neural networks have been able to generalize beyond the training data.  This remains an ill-understood phenomenon.  We do not know under what circumstances such generalization can reliably occur, and I believe any such claims about these generalizations must be validated with independent data sets.  Regardless, however, I am in broad agreement with the authors' views, including their conclusion that machine learning methods have potential for high impact "so long as outcomes are held to the long-standing critical standards that should guide studies of flow physics."

Reference

M. P. Brenner, J. D. Eldredge, and J. B. Freund, 2019:  Perspective on machine learning for advancing fluid mecanics.  Physical Review Fluids, 4:  100501 (7 pages).