Purpose: Students will explore similarities and differences in how scientific research is reported in a journalistic article versus a primary research paper. Questions focus on how each report covers the main aspects of experimental design.
Procedural overview: After reading a Science News article and the original research that the article was based on, students will work in groups to compare the reports by answering the first set of questions provided. Then, students will pool their findings with others in the class for a broader view of the similarities and differences between the two reports. This activity focuses on the Science News article “Hermit crabs are drawn to the dead,” but it can be adapted for any news article based on a primary research paper. Not every question will apply to every set of reports.
Approximate class time: 1 class period.
Getting Source Savvy student activity guide
Pads of sticky notes in at least two colors
Directions for teachers:
Divide the class into five groups (I to V) and assign each group one step of the experimental design process. Steps include:
I. Determining an initial focus by choosing an answerable question and conducting background research.
II. Developing a hypothesis from the proposed question, defining variables and determining the kind of data that needs to be collected.
III. Determining a method to evaluate data collected and the potential errors that will need to be controlled for by the procedure.
IV. Developing and performing a procedure to collect the necessary data.
V. Analyzing and presenting the results.
If needed, please review concepts including scientific method, hypothesis, independent and dependent variables in experiments, experimental controls, experimental error, and statistical analysis of data.
Each group should answer the questions provided below for the assigned experimental design step on sticky notes, with one color for the Science News article and one for the primary research paper. Once groups have answered the questions, indicate where they should stick their answers so that other students can walk around the room and read the notes.
Once all answers are posted, have students pair up with a partner from a different group and take a gallery walk to read their classmates’ answers. During the walk, students should think about the general similarities and differences between the two reports for each experimental design step, including their own assigned step. Once students are finished with the walk, have them head back to their seats and collaboratively write the answers to the gallery walk questions below.
Directions for students:
The Science News article “Hermit crabs are drawn to the dead” reports on the findings of the primary research paper “Scent of death: Evolution from sea to land of an extreme collective attraction to conspecific death,” published open access in Ecology and Evolution.
In this activity, you will explore how specific steps of experimental design are reported in a journalistic article (Science News) versus a primary research paper (Ecology and Evolution). Your teacher will divide the class into groups and assign one step of the experimental design process to each group; you can find the questions for your group below. Answer the questions on sticky notes, with one color for the Science News article and one for the primary research paper.
Once your group has answered its questions, your teacher will tell you where to place your sticky notes throughout the classroom. Then, you’ll take a gallery walk with a partner from a different group to read your classmates’ answers. As you walk, think about the general similarities and differences between the two reports for each experimental design step, including your own assigned step. Back at your seats, you and your partner will collaboratively write the answers to the second set of questions below.
Group I. Determining an initial focus by choosing an answerable question and conducting background research.
1. What background information is given about the research topic?
Science News: As they grow, hermit crabs must take over larger and larger snail shells. The crabs can remodel the shells if necessary, but it is often faster and less taxing to use shells that other hermit crabs have already remodeled.
Ecology and Evolution: The death of one organism frees up biological, physical and social resources for other organisms. Species that transitioned from sea to land had to adapt to a very different environment, and therefore may have changed how they compete for limited resources.
2. Where and to what extent is the background information presented by the author?
SN: In the middle of the article, the author briefly explains the background information.
EE: Section 1, the Introduction, gives extensive background information, citing over 25 previously published articles.
3. What general level of scientific vocabulary is used to explain the background information? Give a specific example from each article.
SN: General vocabulary that would be widely understood even by nonscientists. SN uses “snail” and “another land-dwelling hermit crab.”
EE: Specialized vocabulary that would be understood by other scientists working in the same or similar fields. EE uses “gastropod” and “conspecifics.”
4. What overall scientific question was the research addressing?
SN: The Science News article does not report on the original research question. From the context, the reader might infer that the researchers were trying to find out how attracted hermit crabs are to the scent of their own dead, and whether that attraction is different for sea-dwelling versus land-dwelling hermit crabs.
EE: How do land-dwelling and sea-dwelling hermit crabs respond differently to the resources freed up by the death of another hermit crab?
Group II. Developing a hypothesis from the proposed question, defining variables and determining the kind of data that needs to be collected.
1. What quantitative data was collected to address the research question? Give an example.
Science News: Numbers of hermit crabs attracted within 5 minutes to hermit crab flesh and snail flesh.
Ecology and Evolution: Numbers of hermit crabs within a 30-centimeter radius before and 5 minutes after the introduction of any of five different chemical cues, on land or in the water.
2. What qualitative data was collected to address the research question?
SN and EE: Not discussed.
3. What experimental variables were used in the research?
SN: Type of flesh (crab versus snail) and number and type of attracted hermit crabs (land-dwelling versus sea-dwelling).
EE: Five types of chemical cues, two locations (marine and terrestrial), the number of hermit crabs within a 30-centimeter radius before and 5 minutes after the introduction of the chemical cue.
4. Identify the factor(s) or variable(s) that were manipulated — the independent variable. Identify the factor(s) or variable(s) that were measured — the dependent variable.
SN: Independent variables: type of flesh. Dependent variables: number of attracted hermit crabs, land-dwelling and sea-dwelling.
EE: Independent variables: Five types of chemical cues and two locations (marine and terrestrial). Dependent variable: number of hermit crabs within a 30-centimeter radius before and 5 minutes after the introduction of the chemical cue.
5. Identify the hypothesis that was tested in the research. Are you able to tell if it’s non-directional or directional? Explain.
SN: Does not give details about what the researchers predicted. The article does say that the fact that land-dwelling hermit crabs found hermit crab flesh more appealing than snail flesh and sea-dwelling hermit crabs showed no difference “makes sense.”
EE: Hermit crabs should be attracted to flesh indicating an available shell. But the researchers predict that only terrestrial hermit crabs would be more attracted to the scent of hermit crab flesh (indicating a shell that has been remodeled) than to gastropod flesh. The hypothesis is directional, specifying how the independent variables should affect the dependent variable.
Group III. Determining a method to evaluate data collected and the potential errors that will need to be controlled for by the procedure.
1. How was the data evaluated? Were analysis techniques discussed?
Science News: Results were compared for 20 plastic tubes with hermit crab flesh. Statistical tests were not discussed in any detail.
Ecology and Evolution: Numbers of crabs before and 5 minutes after the introduction of chemical cues were compared for 20 experiments each with five different types of chemical cues in two settings (marine and terrestrial). Statistical analyses were conducted using JMP Pro software to evaluate the mean and standard error of the mean, for example.
2. Name and define at least one statistical test used.
SN: Not discussed.
EE: One-way ANOVA (analysis of variance) testing determines whether there are statistically significant differences among the mean values for different experimental groups (such as those with different types of chemical cues).
3. How did researchers try to minimize errors? Explain.
SN: Not discussed.
EE: Many potential errors were discussed and carefully avoided in the design of the experiments. Hermit crabs were counted before the chemical cue was introduced to account for variations in the initial numbers of hermit crabs. Each test site was at least 5 meters away from any previously tested site, to avoid influence from previous tests. The experimenter left the test site to avoid disturbing the crabs. Students may be able to think of potential errors that were not specifically addressed in the paper.
Group IV. Developing and performing a procedure to collect the necessary data.
1. Briefly describe the procedure used to conduct the research.
Science News: Plastic tubes with hermit crab or snail flesh were set out on the beach and the numbers of hermit crabs (sea-dwelling or land-dwelling) attracted within 5 minutes were counted.
Ecology and Evolution: Tubes on land or in the water were loaded with different chemical cues, and the numbers of hermit crabs within a 30-centimeter radius before and 5 minutes after the introduction of the chemical cues were counted.
2. What experimental conditions and controls were used?
SN: Not discussed.
EE: Land versus sea. Five conditions for each: trypsin (control), live gastropod (control), pure gastropod flesh, trypsin-treated gastropod flesh and conspecific hermit crab flesh.
3. How many trials were run? Be as specific as you can be.
SN: Not discussed.
EE: 100 trials for marine hermit crabs and 100 trials for terrestrial hermit crabs (approximately 20 for each of the five chemical cues).
4. Did the procedure try to minimize potential errors and create reproducible results? Explain.
SN: The article mentions that 20 identical tubes were put out and that data were collected on large numbers of hermit crabs. The article does not discuss whether the results are reproducible.
EE: Yes, Section 2 describes exactly when, where and how the experiments were conducted so that other researchers can replicate the experiments. It also describes how averages were taken over groups of 20 trials per condition to minimize random fluctuations in the results of individual trials.
5. Was the procedure used safe and ethical? How do you know?
SN: Not discussed.
EE: The approval is discussed in Section 2.2. The work was approved by the Costa Rican Ministry of Environment and Energy and by Dartmouth College.
Group V. Analyzing and presenting the results.
1. How was the data displayed? Be as specific as possible.
Science News: In print, a picture of a hermit crab is used to give the reader a general visual of a land-dwelling hermit crab. Online, a video clip of swarming hermit crabs shows an experimental trial.
Ecology and Evolution: Bar graphs in Figures 2 and 3. Figure 2 shows the attraction to the scent of death on land versus in the sea. Figure 3 shows the attraction on land to conspecific versus closely-related heterospecific death.
2. How and where were the results stated in each article? Was the overall hypothesis correct or incorrect?
SN: The results were briefly mentioned near the beginning and near the end of the article. Terrestrial hermit crabs were highly attracted by hermit crab flesh, and to a lesser extent by snail flesh. Marine hermit crabs were roughly equally drawn to hermit crab flesh and snail flesh.
EE: The results are explained in detail in Section 3 and Figs. 2 and 3. The hypothesis was correct. Terrestrial hermit crabs were highly attracted by hermit crab flesh, and to a lesser extent by gastropod flesh. Marine hermit crabs were roughly equally attracted by hermit crab flesh and gastropod flesh.
3. How was the experimental error accounted for? Explain.
SN: Not discussed.
EE: By measuring whether there were any significant differences in conditions between the experiments (weight of flesh, etc.) and by statistically analyzing the results from large numbers of experiments.
4. What didn’t go as planned in the experimental process? How did these errors affect the results? What could be done in the future to minimize unwanted errors?
SN and EE: Not discussed.
5. What additional experiments might researchers want to conduct in the future based on the results of this study?
SN: Not discussed.
EE: Broader comparative research could look at other organisms that rely on rare and valuable resources that might only become available after a death, such as other species with resource-limited dwellings like burrows.
Gallery walk questions:
1. What aspects of experimental design are covered in both reports? Give an example. What are the major differences in how experimental design is covered in the articles? Give an example.
Both articles explain that plastic tubes with bits of flesh were put out to attract hermit crabs, but EE goes into far more detail than SN. There is also general background and a robust results section given in both articles, but again the EE article goes into far greater detail.
The SN article briefly mentions that the scientists put out 20 plastic tubes with bits of hermit crab flesh on a beach to see what they attracted. Many aspects and details are left out. Also, statistical analysis isn’t discussed and data is not displayed graphically.
The EE article explains in detail the locations, times and numbers of experiments, and exactly how the experiments were conducted and analyzed so that other scientists can reproduce the tests. Data and statistical tests are displayed and discussed in detail, so the validity and appropriateness of the analyses can be assessed by other scientists.
2. Who is the author and what relationship does he/she have with the research experiment performed? How does the author’s relationship affect how the content is presented?
Yao-Hua Law, the author of the SN article, is a science journalist not connected to the experiment. Leah Valdes and Mark Laidre are scientists who performed the experiment, and coauthored the EE article.
Journalists report on research findings based on reading primary research papers and related articles, and/or attending scientific talks. Journalists typically interview the scientists who publish the primary research papers and other scientists who have expertise in the area of research to get a nonbiased view of the research. A journalist may summarize the data presented in a research paper, but the journalist does not generate the data.
Scientists who author a primary research paper have had some part in researching, designing, preforming and/or analyzing the data collected. They are intimately involved in at least some aspect of the experiment and outline the experimental process in extreme detail.
3. What audience is the author writing for? Based on the intended audience, what is the overall goal of each report? How does the goal of the report influence what material is presented, including the depth of the experimental design?
The SN article is written for interested members of the public, including students and teachers. The EE article is written for other scientists.
The goal of the SN article is to inform the public about a new scientific finding. The article provides general background information to demonstrate the relevance of the research and a very general procedural overview so readers understand how the results were generated. The main focus of this article is sharing the general results that were found in the research and explaining why they might be important to the broader world.
The goal of the EE article is to inform other scientists about a new scientific finding and the technique by which it was found. Information given in this article should be detailed enough for other scientists to fully understand the experimental design process, including the full procedure, statistical analyses used and results found. Primary research articles need to be able to be fully reviewed by scientists not involved directly in the research, so the scientific findings can be evaluated, reproduced or referenced. Publishing scientific findings and techniques is a way for scientists to contribute new knowledge to the scientific community and get credit for it.