Lab tests aren’t the answer for every science question

In the second half of the 17th century, the chemist and polymath Robert Boyle and philosopher Thomas Hobbes engaged in a divisive debate centered on a temperamental, mechanical contraption known as an air pump. In a series of famous experiments, Boyle used the air pump, which has been called “the cyclotron of its age,” to test basic scientific principles such as the relationship between a gas’s pressure and its volume. But the debate was about more than the scientific results; it was about the very nature of science. As Simon Schaffer and Steven Shapin recount in their 1985 history of the controversy: “Robert Boyle maintained that proper natural philosophical knowledge should be generated through experiment and that the foundations of such knowledge were to be constituted by experimentally produced matters of fact. Thomas Hobbes disagreed.”

To question the value of experimentation in science would be heretical among most modern scientists. So much of today’s effort to understand nature depends on properly designed experiments that control for unwanted variables and are detailed enough to be reproducible. In any high school physics class, students drop objects from the same height and clock how fast they fall to understand gravity, or pull various objects across a table to calculate coefficients of friction. Controlled experiments are even popular in studying the messiness of animal behavior: The chimpanzee tests described by writer Bruce Bower examine how social status affects the spread of a food-snagging strategy.

But controlled experiments still have plenty of limitations. Researchers must have a deep knowledge of their systems and the potential variables at work — a near-impossible task for many areas of inquiry. Assumptions have to be made, about whether a setup really mimics the real world and whether subjects will respond naturally in a lab environment. In “Smartphones may be changing the way we think,” freelance writer Laura Sanders describes some of the troubles in trying to understand how smartphones are changing people’s brains. “This is a lot like drawing conclusions about the effects of baseball on players’ brains after observing three swings in the batting cage,” says experimental psychologist Andrew Przybylski.

In some cases, experimental tests can’t be done because they are unethical. Other times, tests don’t turn out as expected: Two crafty chimps in the food-snagging study found a way to game the system that probably delighted the scientists but also added fuzziness to the results.

In short, experimentation is not always the answer — and certainly not the only answer. In this modern era, with more complicated instruments and more clever testing regimes, we should remind ourselves of the value of observational work. Some of this issue’s most surprising stories rely on data straight from the natural world. Researchers have found, for example, Canadian fossils that might represent the oldest known life. Other scientists uncovered evidence that the Amazon isn’t quite as wild as we might think. By placing monitoring devices on the trunks of just a couple of elephants in Africa, a team discovered that some pachyderms sleep just two hours a day. Try studying that in the lab.