A review of "Farewell to Reality" by Jim Baggott

Farewell to Reality:  How Modern Physics Has Betrayed the Search for Scientific Truth, by Jim Baggott (Pegasus Books, 2013).

The author has an axe to grind with modern physics.  On television and in books about contemporary physics intended for general audiences, established knowledge is seamlessly presented along with speculation and theories (like string theory) which do not, and possibly cannot, have experimental or observational support.  Baggott makes a distinction between what he calls the “authorized version” (theories of physics with well-established empirical support) and “fairy-tale physics” (theories that lack such support).  Moreover, according to him, some physicists have advocated a “post-empirical” re-defining of the scientific method, which would cut science loose from its empirical grounding.

The book begins with a chapter on some amateur philosophy of science, where Baggott sets out the six principles that he thinks demarcate science from metaphysics.  The first is the “reality principle” which is a statement of metaphysical realism – the real world is “out there” independent of our perception of it – tempered by acknowledging that we only have access to “things as they are measured”, not “things in themselves”.  Moreover, “reality is rational, predictable and accessible to human reason.”  Second is the “fact principle” which states that facts are not theory-neutral:  “Observation and experiment are simply not possible without reference to a supporting theory of some kind.”  Third is the “theory principle” which states that any creative process used to develop a theory is acceptable as long as the resulting theory works.  How we define whether a theory works leads to the fourth principle, the “testability principle”, which states that scientific theories must be empirically testable, and for this to be possible auxiliary assumptions are required.  Moreover, no single test is decisive, since either the theory or an auxiliary assumption may be responsible for any discrepancy.  The fifth principle is the “veracity principle” which states that theories can at best be tentatively accepted, while absolute certainty is beyond reach.  The final principle is the “Copernican principle” which states that we are not privileged observers (discussed in a different context by Adams and Laughlin, 1999).  The rest of the book is divided into two parts.  The first is an exposition of the “authorized version”, and the second is titled “The Grand Delusion”, where he outlines “fairy-tale physics” and his problems with it.

Part One begins with a chapter on quantum theory, including the foundational questions.  This is followed by a chapter on quantum field theory and the standard model of particle physics, up to and including the discovery of the Higgs boson.  The next chapter tackles special and general relativity.  Then follows a chapter on the standard model of big bang cosmology, including the inflation model and the unknown nature of dark matter and dark energy.  The final chapter of Part One is about the gaps and flaws of the authorized version.  These include puzzles about quantum measurement, difficulties with the standard models of particle physics and cosmology, and the lack of a theory of quantum gravity.  Efforts to address these issues, such as dark matter searches, are discussed.  Finally, the “fine-tuning problem” is introduced:  this states that the free parameters of the universe seem unusually fine-tuned to allow for the existence of life forms to observe it.

Part Two begins with a chapter on supersymmetry (SUSY).  Baggott feels that SUSY is at least a testable theory and that we can expect experimental elucidation in the next few years.  On the other hand, he is a skeptic of SUSY because he thinks it creates just as many problems as the ones it solves.  He also points to the lack of experimental or observational evidence for supersymmetry thus far, although in my view this judgment is premature.  The next chapter takes on the numerous flaws of string theory (including superstrings and M-theory), ground previously trodden most famously by Smolin (2006) and Woit (2007).  The next chapter tackles various versions of the multiverse concept, from the “many worlds” interpretation of quantum theory, to the inflationary multiverse.  All of these are dangerous, in Baggott’s view, as they violate the testability principle.  The next chapter, “Source Code of the Cosmos,” tackles a hodge podge of ideas.  The first is Max Tegmark’s claim that the universe is a mathematical structure.  Next, he presents quantum information theory and quantum computing, in a more non-committal way.  (I assume he believes these fields do fall into the legitimate side of science, though not yet part of the authorized version, since much remains to be worked out in those fields.)  He then discusses the “black hole war” (involving ideas from general relativity, quantum theory, thermodynamics, and quantum information) which was resolved by Juan Maldacena’s holographic principle.  The fact that the “black hole war” between Stephen Hawking and Leonard Susskind was finally resolved shows that progress can be made here, but it is not the kind of progress Baggott would prefer.  The resolution of the “war” was based entirely on theoretical developments, without grounding in observational or experimental data.

The book’s penultimate chapter takes on the anthropic cosmological principle, which directly contradicts the Copernican principle that Baggott develops at the start of the book.  He also takes a swipe at the John Templeton Foundation in this chapter.  In the concluding chapter, Baggott tries to answer six questions.  First, “If fairy-tale physics isn’t science, what is it?”  Baggott’s answer is that the stuff isn’t even metaphysics, but rather “nothing but sophistry and illusion” (quoting philosopher David Hume).  Second, “But aren’t theoretical physicists supposed to be really smart people?”  He answers affirmative but gives an analogy with the financial crisis of 2008, which was partly the result of very intelligent financial engineers who nonetheless fell under a “grand delusion”.  Third, “Okay, but in the grand scheme of things is there any real harm done?”  Baggott’s answer is that the “integrity of the scientific enterprise” is being harmed.  This is where he trots out Brian Greene and Leonard Susskind apparently defending a post-empirical redefinition of the scientific method.  Fourth, “What do the philosophers have to say about it?”  Baggott cites only a commentary by philosophers Cartwright and Frigg (2007), but otherwise would like to hear more from philosophers.  Baggott states that “the guardianship of science and the scientific method should not be left solely in the hands of scientists, particularly those scientists with intellectual agendas of their own.”  Fifth, “Are we witnessing the end of physics?”  Baggott cites Horgan (1996) but offers that the list of unanswered questions in physics is still quite lengthy.  The real problem is impatience, which Baggott feels is a factor driving the development of fairy-tale physics.  The final question is “So, what do you want me to do about it?”  Baggott’s answer is to maintain a healthy skepticism when reading about contemporary physics.

So, what to make of the book?  Baggott focuses on particle physics, cosmology, and quantum information theory.  He makes no reference at all to the largest field in physics, condensed matter, not to mention all the other subfields of physics.  Smolin (2006) does the same but at least explains that he does; Baggott never explains that there are vast areas of physics untouched by the “fairy tale” issue he rants about.  Baggott also fails to explore the sociological reasons why “fairy tale” physics persists, an issue that Smolin (2006) does address in some detail.  Thus in comparing the two books, Baggott tackles a broader set of issues (whereas Smolin is mainly concerned about string theory) but Smolin gives a much more thorough account of his topic.

Personally I think Baggott is mostly right (though his dismissal of SUSY due to lack of evidence is far too premature).  However I think Smolin does a better job of convincing us that fairy-tale physics is actually damaging—how funding and hiring is being dominated by less than worthy theoretical efforts.  Baggott is clearly ticked off, but is not articulate enough about the damage and why we should care.  I am not as prepared to completely dismiss string theory as Baggott and Smolin are, but I certainly agree that in their current form they offer little in the way of scientific progress.  Nonetheless, it’s about time someone wrote a book like Baggott’s.

References

Fred Adams and Greg Laughlin, 1999:  The Five Ages of the Universe:  Inside the Physics of Eternity.  Free Press.

Nancy Cartwright and Roman Frigg, 2007:  String theory under scrutiny.  Physics World, Sept. 2007, p. 15.

John Horgan, 1997:  The End of Science:  Facing the Limits of Knowledge in the Twilight of the Scientific Age.  Little, Brown.

Lee Smolin, 2006:  The Trouble with Physics:  The Rise of String Theory, the Fall of a Science and What Comes Next.  Penguin.

Peter Woit, 2007:  Not Even Wrong:  The Failure of String Theory and the Continuing Challenge to Unify the Laws of Physics.  Vintage.

Surface tension and biology

This blog has tended to focus on methodological issues in science and medicine, but occasionally I do want to lavish praise for substantive work.  Here I'd like to call readers' attention to the delightful article in Science a couple weeks ago by Elizabeth Pennisi, "Water's Tough Skin."  It is a feature article describing a number of ways that surface tension is important in biology, including for plants, animals, and microbes.  A number of scientists, engineers, and mathematicians are interviewed about their work.  This is the kind of article that reminds us of why we became interested in science, engineering, and medicine in the first place.  I won't review the content here, but I commend it to readers to enjoy for themselves.

Reference

Elizabeth Pennisi, 2014:  Water's tough skin.  Surface tension is a force to be reckoned with, especially if you are small.  Science, 343:  1194-1197.

The perils of sharing data

One of the cardinal principles of reproducible research is the sharing of data.  Making data (not just a summary of results) available is just as important for disclosure as the study design, and materials and methods.  Disclosure of the actual data permits others to confirm the analysis of the original authors, or conduct alternative analyses.

Last week's issue of Nature had an editorial outlining the perils of sharing data.  Of greatest concern:  once a data set is published, it is easy for an armchair analyst to take it and run with it, possibly depriving the original investigators the opportunity to publish findings based on a very hard won data set.  Publication of a data set does not have the same status and prestige as publishing scientific findings from the data.

I join with Nature in encouraging discussion and debate within the scientific community to resolve these issues.  The interests of the innovators -- those who design studies and collect the data -- need to be considered before rushing into a mandatory data disclosure policy.  The article talks about the possibility of inviting the original authors to be co-authors on works derived from their data by others.  This is one of many possible options.  The infrastructure of science, including funding, tenure, and promotion policies, needs to change to accommodate and encourage data sharing.  I don't have any answers, and I suspect each field will have to structure a customized solution to suit its own situation.

Welcome to Ioannidis & Goodman's METRICS

Last week's edition of the Economist carried an article announcing the creation of a new laboratory at Stanford University, called the Meta-Research Innovation Center (METRICS), founded by John Ioannidis and Steven Goodman.  It aims to promote reproducible research and shame non-reproducible research.  Of greatest note is its "journal watch" concept, which will monitor the quality of published research.  Such a watchdog has been badly needed for decades, although I wonder how the operation will be funded.

Other activities mentioned by the article include influencing policymakers to avoid relying on shoddy research, battling publication bias, organizing conferences of other meta-researchers, and evaluating the effects of encouraging reproducible research. In other words, they'd like to see if there is any evidence that evidence-based science is better science!

DTLR has long been an admirer of Ioannidis' work, and I welcome the new center and look forward to seeing its work.

This winter's weather does not reveal anything about global warming

Last month in Science, a group of five prominent atmospheric scientists from around the country published a letter cautioning us not to interpret this winter's severe weather through the lens of climate change (Wallace et al., 2014). This winter has witnessed the dip in the polar vortex over much of the U.S., resulting in all time low temperatures; there have also been a series of heavy snowstorms on the east coast. Speaking of these events, the authors write:

Some have been touting such stretches of extreme cold as evidence that global warming is a hoax, while others have been citing them as evidence that global warming is causing a “global weirding” of the weather. In our view, it is neither.

As climate scientists, we share the prevailing view in our community that human-induced global warming is happening and that, without mitigating measures, the Earth will continue to warm over the next century with serious consequences. But we consider it unlikely that those consequences will include more frigid winters.

Although such a hypothesis has been proposed, the authors do not find it corroborated with either “alternate observational analyses” nor climate model simulations. Moreover they “do not view the theoretical arguments underlying it as compelling.” The authors caution about mistaking coincidence for causation, and although they believe such hypotheses “deserve a fair hearing”, the authors seem to imply that this one is too half-baked to be made “the centerpiece of the public discourse on global warming.” They conclude:

Even in a warming climate, we could experience an extraordinary run of cold winters, but harsher winters in future decades are not among the most likely nor the most serious consequences of global warming.

Reference

John M. Wallace, Isaac M. Held, David W. J. Thompson, Kevin E. Trenberth, and John E. Walsh, 2014: Global warming and winter weather. Science, 343: 729-730.

A call for more reproducible research in drug discovery/development

Phase III drug trials are typically randomized, blinded, controlled clinical trials.  However, last month in JAMA, Djulbegovic et al. (2013) argued "more than 80% of phase 1 studies and more than 50% of phase 2 studies are currently nonrandomized."  They argue that these early phase studies should all be randomized, and that even preclinical studies in animals and cell cultures should also be randomized.  (Randomization is probably even less prevalent in preclinical research than in the early phase studies discussed in the quote.)  In other words, the authors advocate study designs that encourage reproducibility across the spectrum of clinical and preclinical research.

They argue that non-randomized studies can easily lead to incorrect decisions, both pro and con.  Thus randomized studies are more efficient and provide stronger backing for decision making.  (They also make the case that randomized studies are more ethical; I'm not sure I find their reasoning here as compelling.)  They hope that use of more rigorous study designs across the drug development arena could be one way to address the industry's infamously high failure rate.

Here is a key passage from the article, describing the literature in preclinical research.

This literature yields an excess of statistically significant findings that cannot be eventually replicated, let alone translated into clinical successes.  For preclinical research conducted by the industry, routine adoption of rigorous randomized designs should be straightforward--no company wants to spend millions of dollars for the clinical testing of useless treatments.  In fact, industry researchers have taken the lead in raising the concerns about the reproducibility of preclinical research and suggesting partial solutions.  For preclinical research conducted by non industry researchers, similar rigorous practices can also be routinely adopted and requested.  Funders and journals can specify that they will sponsor and publish animal studies only if they fulfill rigorous randomization criteria.  Justified exceptions to this rule are likely to be rare.

(I have not included the footnotes; see the original.)  The major lesson for me in the above passage is that the main contribution of statistics to such studies is in the design, not the analysis.  "Statistically significant" findings by no means guarantee that the study has a chance of being reproduced, whereas good study design principles would greatly enhance the likelihood that such studies are reproducible.  Unfortunately much of the teaching and practice of statistics, both by statisticians and non-statisticians, tends to emphasize the mathematical/calculational side, rather than the study design side.

I'd like to see a devil's advocate's response to all this.  I find the authors' views compelling, and have difficulty imagining the grounds for which one might disagree.

Reference

Benjamin Djulbegovich, Iztok Hozo, and John P. A. Ioannidis, 2013:  Improving the drug development process:  more not less randomized trials.  Journal of the American Medical Association, 311 (4):  355-356.

Responses to "When Mice Mislead"

This past week's issue of Science (the Jan. 24, 2014 issue) has two letters to the editor, responding to a report last November, "When Mice Mislead" by Jennifer Couzin-Frankel, which I discussed in an earlier post.  The first letter, by Richard Traystman and Paco Herson, points to earlier findings, similar to those reported by Couzin-Frankel, in the stroke research community.  Most importantly, they assert that "It is unlikely that poor methods used in animal studies account for all the negative clincial trials that have been performed based on preclinical studies.  After all, some investigators do perform appropriate experiments, and even those studies rarely lead to positive clinical trials."  The authors point to the fact that mouse studies are usually done with healthy young mice, whereas human subjects in neuroprotective drug clinical trials are often older and have many co-morbidities.  They propose that aged mice with comorbid diseases be used in stroke trials, as a better animal model of human disease.

The second letter is from statistician Gary Churchill.  He zeroes in on one key question:  "Was the result replicated in more than one genetic background?"  He goes on to identify two "root causes" for nonreproducible research:

Science today is driven by an incentive system that often rewards precedence and impact over quality of the work.  Statistical training of scientists often emphasizes analytical techniques over experimental design and quantitative reasoning.  These are systemic problems that will not change without substantial effort

Meanwhile, Churchill endorses the message of Couzin-Frankel's article with his maxim:  "Be wise, randomize."

I think that both of these letters add value to the original piece by Couzin-Frankel. In particular, Churchill's second "root cause" is particularly interesting, as both statisticians and lay scientists or mathematicians who teach statistics are all guilty of overemphasizing methodology, modeling, and inference at the expense of study design and critical thinking. 

References

Jennifer Couzin-Frankel, 2013: When mice mislead. Science, 342: 922-925.

Richard J. Traystman and Paco S. Herson, 2014:  Misleading results:  translational challenges.  Science, 343:  369-370.

Gary Churchill, 2014:  Misleading results:  don't blame the mice.  Science, 343, 370.