For all his other talents, Charles Darwin wasn’t much of a mathematician. In his autobiography, he writes that he studied math as a young man but also remembers that “it was repugnant to me.” He dismissed complex mathematical arguments and wrote to a friend, “I have no faith in anything short of actual measurement and the Rule of Three,” where the “Rule of Three” was an extremely simple mathematical calculation.
But history played a joke on the great biologist: It made him a contributor to the development of statistics.
It was the wildflower common toadflax that got the whole thing started. Darwin grew the plant for experiments, and he carefully cross-fertilized some flowers and self-fertilized others. When he grew the seeds, he found that the hybrids were bigger and stronger than the purebreds.
He was astonished. Although he had always suspected that inbreeding was bad for plants, he had never suspected it could have a significant effect within a single generation.
So he repeated the experiment with seven other kinds of plants, including corn. He had a clever, and at that time novel, idea. Since slight differences in soil or light or amount of water could affect the growth rates, he planted the seeds in pairs — one cross-pollinated seed and one self-pollinated seed in each pot. Then he let them grow and measured their heights.
Sure enough, on average, the hybrids were taller. Among his 30 corn plants, for example, the purebreds were only 84 percent as tall as the hybrids. But Darwin was savvy enough not to simply trust the average heights of so few plants. “I may premise,” Darwin wrote, “that if we took by chance a dozen or score of men belonging to two nations and measured them, it would I presume be very rash to form any judgments from such small numbers on their average heights.” Could it be, he wondered, that the height differences in the plants were just random variation?
Darwin noted, though, that men’s heights vary a lot within a single country, whereas the heights of his plants didn’t. His result might be more meaningful, but he wanted to be able to quantify how meaningful.
Doing that, however, required Darwin’s hated mathematics.
So he turned to his cousin, Francis Galton, who just happened to be a leader in the emerging field of statistics. Galton had recently invented the standard deviation, a way of quantifying the amount of random variability in a set of numbers.
But Galton wasn’t all that much use. He could calculate the standard deviation, but he couldn’t use that number to tell Darwin how likely it was that the height difference wasn’t just random. Furthermore, he was pretty sure it was too few plants to tell. “I doubt,” he wrote, “after making many trials, whether it is possible to derive useful conclusions from these few observations. We ought to have measurement of at least fifty plants in each case, in order to be in a position to deduce fair results.”
And there the matter rested, in frustrating uncertainty, for 40 years.
Resolving the impasse, it turned out, required some beer. The Guinness brewing company hired a young University of Oxford graduate, William Sealy Gosset, to develop statistical techniques to cheaply monitor the quality of its beer. The method Gosset developed was so powerful that it transformed statistics and continues to be a workhorse to this day.
Ironically, though, Gosset wasn’t allowed to publish the method under his own name, because Guinness wanted to keep it a secret that statistics could help make better beer. But publish it he did, under the pseudonym “Student.” The technique has hence become known as the “Student’s t-test.”
The Student’s t-test did just what Galton didn’t know how to do: Given the standard deviation Galton had calculated, it told how likely it was that the difference in the heights between the hybrids and the purebreds were just random. The answer? The chance was about one in 20. By statistical standards, that’s significant, but barely so.
It took another 10 years and the intervention of another statistical genius for the next breakthrough on the problem. As a college student, Sir Ronald Aylmer Fisher learned about Gregor Mendel’s work in genetics and Darwin’s work in evolution, but the theory connecting the two hadn’t yet been developed. Fisher set out to create the statistical foundation to make the connection possible. Darwin’s experiment with hybrids was just the kind of problem Fisher needed to be able to solve.
He noticed something that Galton had missed: Galton had ignored Darwin’s clever method of pairing the plants. He had calculated the standard deviation of the plants as a single, large group.
Fisher repeated the analysis but calculated the standard deviation of the difference in heights between the pairs of plants in each pot. Suddenly, instead of a one in 20 chance that the result didn’t mean anything, he calculated about a one in 10,000 chance. In other words, it was nearly certain that the hybrids really did grow taller than the purebreds.
Fisher noted that the Student’s t-test had one possible flaw: It assumed that the plant heights would vary in a predictable way (according to a normal distribution, to be precise). Just in case that assumption was wrong, he devised another way of analyzing the data and confirmed the result. “He was very clever in the way he did it,” says Susan Holmes of Stanford University. Only in the 1980s did statisticians realize the full potential of Fisher’s method and develop it into the subject of “exact testing.”
Fisher’s analysis was only possible because Darwin had designed his experiment so well. In fact, Fisher was often frustrated with the quality of other people’s experiments. “To call in the statistician after the experiment is done,” he said, “may be no more than asking him to perform a postmortem examination: he may be able to say what the experiment died of.”
David Brillinger, a statistician at the University of California, Berkeley, says that Darwin’s method of pairing is now common practice. “Darwin was a leader in a subfield of statistics called experimental design,” he says. “He knew how to design a good experiment, but what to do with the numbers was something else.”
Darwin himself came around eventually in his attitude toward mathematics. While he wrote in his autobiography of his youthful distaste for math, he also wrote that he wished he had learned the basic principles of math, “for men thus endowed seem to have an extra sense.”
