A review of "Research Methods for Science," by Michael Marder. Part 2. Randomization.

In this post I continue my review of Michael Marder’s book, Research Methods for Science (Cambridge University Press, 2009).  I want to dwell here on the concept of randomization, which has many applications in the design of experiments (Peirce & Jastrow, 1885).  For instance, random assignment of subjects to treatments, in a controlled clinical trial, along with double-blinding, helps to rule out competing explanations of the trial outcomes, such as confounding variables and self-selection (which can haunt non-randomized studies).  When there are no systematic differences between treatment groups other than the intervention being evaluated, the results can only be causally linked to either the intervention or to chance.  Similarly in studies where histological samples or medical imaging files need to be scored or segmented by human raters, randomized and blinded presentation of the samples or images should be done to reduce potential systematic errors.  Randomization and blinding together help secure the attribution of causality by ruling out systematic differences between treatment groups other than the intervention under study.

Randomization is never discussed in Marder’s book, despite its hallmark status in comparative experiments, which earns discussion in potentially competing books such as Wilson (1952/1990), Valiela (2009), and Srinagesh (2006).  Marder’s omission is particularly glaring since he promises to show us procedures that avoid bias (p. 4) and later discusses clinical trials (p. 40), where randomization is absolutely essential.  The author does discuss random sampling, in the context of surveys (Sec. 2.3.3), but only hints at the limitations of convenience sampling, the most common type of non-random sampling.  Moreover, he writes that “One way to obtain random lists of student names is to type two-letter sequences into a university directory search engine” (p. 43).  On the contrary, such an exercise is an illustration of haphazard, not random sampling.  The founder of the statistical theory of experimental design, R.A. Fisher, specifically ruled out arbitrary human choice as a source of randomness in his classic 1935 monograph, The Design of Experiments (Fisher, 2003, p. 11).  In contemporary science, engineering, and medicine, there is no excuse for avoiding either pseudo-random number generators (readily accessible in spreadsheets or any decent scientific or statistical software program) or physical sources of random numbers (radioactive decays and suchlike) to produce random assignments or random samples (e.g., Hayes, 2001).

Incidentally, another hallmark principle of experimental design, blocking, appears to be beyond the scope of Marder’s book -- presumably because the analysis of variance is stated to be outside its scope (p. vii).  Competing texts by Valiela (2009) and Srinagesh (2006) do include a discussion of blocking, as does any basic statistical text on experimental design.

Marder’s omission of randomization in a book about how to do science is a fatal oversight.  It is a lost opportunity to introduce good habits and careful experimental procedures.  A discussion of what can go wrong without randomization would have provided opportunity to illustrate critical thinking in science.  Randomization is not an advanced concept, and science undergraduates should be expected to know why it is important and how to carry it out. 

Incidentally, last year a Florida Congressman (who has a bachelor’s degree in electrical engineering) sponsored a bill to end the US Census Bureau’s American Community Survey in part because “in the end this is not a scientific survey. It’s a random survey.”  Go figure!

References

 

 R.A. Fisher, 2003:  The Design of Experiments.  First edition published, 1935; eighth edition, 1966.  Reprinted in R.A. Fisher, 2003:  Statistical Methods, Experimental Design, and Scientific Inference (Oxford University Press).

B. Hayes (2001): Randomness as a resource. American Scientist, 89 (4): 300-304.

C.S. Peirce and J. Jastrow, 1885:  On small differences in sensation.  Memoirs of the National Academy of Sciences, 3:  73-83.

K. Srinagesh, 2006:  The Principles of Experimental Research.  Butterworth-Heineman.

I. Valiela, 2009:  Doing Science:  Design, Analysis, and Communication of Scientific Research.  Second edition.  Oxford University Press.

E.B. Wilson, Jr., 1952/1990:  An Introduction to Scientific Research.  McGraw-Hill, reprinted by Dover.