Cookieology: Experimental design 101

This exercise is a part of Educator Guide: Fight Like an Animal / View Guide

Download Student Activity Guide

Purpose: To learn and practice the initial steps of designing a scientific research project.

Procedural overview: Design a scientific experiment to create the ideal sugar cookie. If time and resources allow, run the experiment, collect and analyze data, and report your results.

Approximate class time: One class period, or more if students will be conducting their experiments.

Supplies:

Computers
Access to the internet
Cookie supplies
- Black+Decker 4-slice toaster oven or other oven at school
- Parchment paper
- Oven thermometer
- Kitchen scale that measures in grams
- Toothpicks
- Ruler
- Oven mitt
- Spatula
- Premade sugar cookie dough or the ingredients and a general recipe

Direction for teachers: This activity is designed to be a fun way to introduce and allow students to explore experimental design. You can easily alter this activity to make it appropriate for any science class type and level, and you can tailor it to the resources that are available to you and your students. Students probably won’t be thrilled if they aren’t allowed any class time to complete their experiment, but if you don’t have the time or resources at school, then you can encourage them to complete their experiment at home while carefully documenting their work and the resulting data.

Use the Science News for Students “Cookie Science” post titled, “Bake your way to your next science project!” to customize the Cookieology experience for your class. There are a total of 18 posts on designing an experiment to see if a gluten-free cookie can hold its own when compared with a flour-based cookie. The articles cover everything from the experimental design, testing, data collection, results, statistics, research, background knowledge and how to summarize and display your results for the cookie question at hand.

Directions for students:

Have you ever wondered why baking cookies smell good? Or why cookies taste good, especially when they are slightly crispy? What exactly happens in the oven to turn dough into a cookie? Does it surprise you to hear that baking a cookie is actually one big science experiment and that each recipe is essentially an experimental procedure?

To explore the steps of experimental design, you will plan an experiment to make an ideal sugar cookie. This lab activity is developed from a number of resources including the Science News for Students Eureka Lab! series, “Bake your way to your next science project!,” the 2017 Research in Brief Educator Guide as well as a lesson plan developed by a Science News in High School educator and “tested” with her own research classes. This activity is designed to walk you through the initial steps of designing and conducting an experiment.

I. Determine an initial focus and conduct background research.

The first step in the scientific experimental design process is to find a good topic in order to formulate a suitable research question and hypothesis. Since this activity defines the scope of the “research field,” you might not need to do additional searching to find a research topic at this time.

In the future, to start your own research project, you would need to find a topic that interests you. Science research project ideas should come from you, not a parent, teacher, research mentor or someone else. There are many ways to find good project ideas. You might explore the real science that is related to something in a science fiction film or story. You might have ideas for an engineering solution to a real-world problem encountered by you or a family member or friend. You might propose a scientific analysis or engineering application based on one of your hobbies or interests, such as bicycling, sewing or filmmaking. You might think of new questions or applications after learning about new science research in Science News and Science News for Students articles. Log in to Science News in High Schools and search the most recent articles or just look through the current issue of Science News. In the student activity section of the Research in Brief Educator Guide, you could answer the related “Generating Interesting Topics and Questions” prompts to help you form ideas.

In order to proceed with designing your ideal sugar cookie, you probably need to shore up your food science and baking 101 knowledge. In this section, spend some time getting to know the ingredients and science behind the cookie baking process.

For more information on conducting background research, see:

Cookie Science 12: Heading to the library.

Cookie Science 13: The deal with gluten.

Here are a few resources to get things started and to refer to along the way:

Science News for Students, “Eureka! Lab: Bake your way to you next science project!”

TED-Ed, “The chemistry of cookies – Stephanie Warren.”

ACS Reactions Video “How to Cookie with Science”

1. After watching one of the listed cookie science videos, do you have any initial thoughts about possible questions you would like to test? List three potential questions below.

Does the amount of leavening agent, or raising agent, used affect the height and width of the cookie?

Can a sugar cookie be made with too much sugar?

Should you add salt to a cookie, and, if so, what is the optimal amount?

2. Do you have a favorite sugar cookie recipe memorized? If not, you’re going to need to do a little research to find a basic recipe that provides a good starting point for your design. Or, if you love the ready-made sugar cookies, then look up the ingredients and cooking instructions for a common type. Print out the recipe or write it below.

Example of a general sugar cookie ingredient list:

1½ cups white sugar, 1 cup softened butter, 1 egg, 1 teaspoon vanilla, 2¾ cups flour, ½ teaspoon baking powder (chemical leavening agent), 1 teaspoon baking soda (chemical leavening agent) and ½ teaspoon salt.

Cooking directions will vary.

3. Assuming you would be using scientific, metric-based lab equipment, do English to metric conversions for each ingredient. (Liquid measurements should be converted to milliliters and solid measurements should be converted to grams.)

Student answers will vary based on the recipe chosen.

4. What are the chemical components of each ingredient? Which ingredients are pure substances or mixtures? Explain your answer and give the chemical makeup of the ingredients — include the chemical formula of ingredients, when appropriate. What role does each ingredient play in cookie composition?

Sugar: Common table sugar, or sucrose, is a pure substance known for its sweet taste. Sucrose is a disaccharide composed of two chemically bonded monosaccharaides, glucose and fructose. When sucrose is heated to approximately 310° Fahrenheit, the Maillard reaction will occur. In this reaction, the carbonyl group of the sugar reacts with the amino group of the amino acid present to produce the signature brown, toasted color and a nutty taste. When sucrose is heated to approximately 350° F, a caramelization reaction will occur — this is when heat breaks down sucrose in the presence of oxygen into the smaller molecules associated with a caramel flavor and other nutty aromas.

Flour: Flour is a mixture. It contains carbohydrates, or starch granules, and proteins such as glutenin and gliadin. Whole wheat flour also contains small pieces of other parts of grain. When water is added to flour, the proteins that were tightly coiled uncoil and link together to form a gluten network. The more you mix the wet flour, the stronger the gluten network becomes. If the gluten network is too strong, gas produced in the baked good will not expand properly. If the gluten network isn’t strong enough to stay risen, the baked good will collapse even if enough gas was produced during the baking process. Flour provides the framework for the general structure of a baked good.

Butter or shortening: Butter is a mixture of fat and water, while shortening is a mixture of only various types of fats, such as vegetable oils. Fats, or triglycerides, create a hydrophobic coating on flour proteins and are used to prevent portions of the gluten network from forming during baking. Slowing gluten formation results in a softer, tenderer cookie.

Egg: Eggs are considered mixtures because they contain many different chemicals and have distinct parts — the yolk and albumen, commonly known as egg white. Eggs help give baked goods a solid structure. When heated to approximately 145° F, the proteins in eggs unwind and create a solid structure. When eggs are added to a dough, the egg proteins coagulate and act as a binding ingredient. Egg whites with air whipped into them can also act as a leavening agent if folded into batter carefully so air does not escape.

Chemical leavening agent: Chemical leavening agents can be mixtures or pure substances. Leavening agents produce gas during the baking process and are therefore used to make batter or dough rise. Baking soda, or sodium bicarbonate (NaHCO₃), is a pure, basic substance of only one compound that will react with acids to create carbon dioxide gas. Baking powder is a mixture of NaHCO₃ and two acids, typically monocalcium phosphate (Ca(H₂PO₄)₂) and sodium acid pyrophosphate (Na₂(H₂P₂O₇)) or sodium aluminum sulfate (NaAl(SO₄)₂). In baking powder, the base NaHCO₃ will not react with the Ca(H₂PO₄)₂to produce carbon dioxide gas until water is present. The second acid, either Na₂(H₂P₂O₇) or NaAl(SO₄)₂, will not react with NaHCO₃to produce carbon dioxide until the wet dough is warm. The second acid in baking powder prolongs the leavening process so that a new reaction producing carbon dioxide gas begins when the cookies are in the oven. When water in the dough is heated, the resulting steam can also act as a leavening agent. Egg yolks also have lecithin, which is an emulsifier. Emulsifiers allow hydrophobic fats and hydrophilic water-based liquids in a recipe to combine uniformly.

Table salt: Table salt is mostly NaCl, but may contain other additive ionic compounds, such as potassium iodide (KI) or sodium phosphate (Na₃PO₄). NaCl alone would be considered a pure substance, but when another ionic compound is added, table salt is considered a mixture. In cookies, table salt helps to round out the flavor profile. (In bread, table salt also helps control the production of carbon dioxide gas by yeast, by drawing the water out of yeast.)

5. Name a few chemical reactions and physical changes that occur when ingredients are mixed together and baked.

Chemical reactions: Maillard reaction, caramelization reaction, formation of a gluten network and all leavening acid-base reactions that produce carbon dioxide gas.

Physical changes: fat coating flour proteins, emulsification using egg yolks and production of steam from water.

6. Drawing on what you have learned from your research of the related scientific concepts, look back at your original questions and modify them or ask a new question that could be answered through observation or experimentation.

Does the amount of baking powder used affect the height of the sugar cookies? Which leavening agent works best? Which type of fat makes the softest sugar cookie?

7. Describe the results you expect to observe.

The more baking powder used, the thicker the cookie will be. Baking soda will be the best leavening ingredient because it will allow carbon dioxide to be produced throughout the baking process. Shortening will produce a thicker cookie than butter, because it does not contain any water.

II. Define your variables and develop your own testable Cookieology hypothesis from your proposed question.

After all that research and brainstorming, it’s your turn to determine how to scientifically address your cookie question. You’ll need to plan out an experiment by first developing a hypothesis that you want to test. Your hypothesis should be testable and should identify variables that can be measured or qualified throughout the experiment. You can collect quantitative data (data that consists of numbers, such as temperature, width or mass) and/or qualitative data (data that consists of nonnumeric characteristics as perceived by a human, such as texture or color).

1. What quantitative cookie data might you want to collect to address your cookie question, and how would you collect it?

Duration that cookies were in the oven (with a timer); actual oven temperature (with one or more oven thermometers); internal cookie temperature (with a cooking thermometer); ingredient masses or cookie total mass (with a scale); cookie width or thickness (with a ruler); cookie softness (using a press to measure the force exerted and a ruler to measure the change in cookie thickness) and pH of the ingredients or cookie (with quantitative colorimetric pH paper).

2. What qualitative cookie data might you collect to address your cookie question, and how would you collect it?

Cookie color (photographed and compared with a chart of colors by the experimenter or by volunteers), texture (judged on a relative scale by taste-tester volunteers or evaluated by the experimenter with photographs, or with a magnifier or a low-power microscope) and taste (judged on a relative scale by taste-tester volunteers).

Now that you have determined what data you can collect, establish your experimental variables and eliminate unwanted ones. A variable is a factor, trait, object or condition whose value can change in the course of an experiment. As you saw from above, a variable can be qualitative (descriptive) or quantitative (measurable). A quantitative variable can be continuous or discrete. Cookie height, for example, is continuous because it can be any number between its minimum and maximum value. The number of heads or tails in coin tosses, though, is discrete because you would only use whole numbers to describe the data.

The independent variable is a factor that you, the experimenter, manipulate in order to observe its relationship to a phenomenon that can be measured (the dependent variable). Think “I” for “independent” and the one that “I” can control. Think “D” for “dependent” and for “data generated.” Experiments are designed to find out how the independent variable affects the dependent variable.

3. Identify one factor or variable that you will manipulate — the independent variable.

The amount of baking powder.

4. Identify one factor or variable that you will measure — the dependent variable.

The height and width of the resulting sugar cookie.

Now, create a testable hypothesis: Your hypothesis will need to be tested to generate one or more lines of evidence that will either support or refute it. A good hypothesis should be original and testable. Think through an experiment that can be done and that is repeatable. Write a hypothesis that defines a relationship between your variables. You may want to narrow down your research problem to a statement that is directional. A non-directional hypothesis defines that a relationship exists between two variables in some way. A directional hypothesis not only defines the relationship between variables, but also predicts a positive or negative change or difference between the two variables.

For examples, see Cookie Science 2: Baking a testable hypothesis.

5. Write a non-directional hypothesis that defines relationships between your variables.

The amount of baking powder will affect the height and width of the resulting sugar cookie.

6. Write a directional version of the above hypothesis.

As the amount of baking powder added increases, the height of the resulting cookie will increase and the width will decrease.

III. Think about how you would analyze and display the data you collect, and the potential errors that will need to be controlled for by the procedure.

For additional information, see: Cookie Science 5: ‘Blinding’ your subjects.

1. How will you plan to analyze your data? Will you run any mathematical tests to see if there is statistical significance among your data, either proving or disproving your hypothesis? If so, make sure you determine the population size that you need to run the specific statistical test. Your teacher will give you more guidance for this question.

Student answers will vary based on class level. Teachers should adjust the question so that it is suitable for their class.

2. Some errors in data collection are random. Random errors can be either positive or negative fluctuations that affect the accuracy of your data. The best way to minimize random error is to test as many identical samples as possible, then take the average of those results. What sorts of random errors might occur in the cookie project?

Random errors could occur because of temperature fluctuations during baking and differences in the masses of each cookie.

3. Other errors in data collection are systemic. They always occur in the same direction; all of the measurements are off in the same way. Systemic errors affect the accuracy of the data. What sorts of systemic errors might occur in the cookie project?

The oven temperature might be consistently higher (or lower) than the set temperature. The weigh scale might not be calibrated, in which case it would always overestimate (or always underestimate) the dough mass of each cookie.

After analyzing the data, charts and graphs can be used to illustrate relationships between variables, or what your data found. Consider the following charts and graphs, and think about which one will most accurately represent the findings of your cookie experiment.

4. Pie charts show each component’s relation to the whole. The categories are represented as slices of the pie and are usually shown in different colors. What cookie data might you graph using pie charts?

Proportions of ingredients in cookies (by mass or by volume) or fractions of taste-tester volunteers who gave various responses about the cookies.

5. Bar graphs compare values between two or more populations, with the height of each bar representing the value for that population. Sometimes single bars are divided into segments of different colors to illustrate the relative contributions of different factors to that group. What cookie data might you graph using bar graphs?

Average diameter of the cookie at different conditions or fractions of taste-tester volunteers who gave various responses about the cookies.

6. Time-series graphs show measurements over a period of time. Either axis could be used for time, but usually time is plotted on the x-axis (horizontal axis). Data are plotted as dots or small circles. To help visualize the trend in the data, data points are usually connected by straight lines between data points or by a smooth curve of a mathematical function that best explains the trend in the data. What cookie data might you graph using time-series graphs?

The spread of the cookie relative to the time spent in the oven. The softness, temperature or other properties of the cookie relative to the time spent in the oven.

7. Scatterplots show a relationship between two quantitative variables. The independent variable is plotted on the x-axis (horizontal axis) and the dependent variable is plotted on the y-axis (vertical axis). What cookie data might you graph using scatterplots?

The change in height or rise of the cookie (on the y-axis) versus the amount of a certain ingredient added to the recipe (on the x-axis). Cookie softness (on the y-axis) versus the amount of a certain ingredient or the baking time (on the x-axis). Oven temperature (on the y-axis) versus location within the oven (in any one dimension, on the x-axis).

IV. Write and perform your procedure.

If your teacher asks, create a stepwise, detailed procedure for your Cookieology research project. While designing the procedure, make sure you think about ways to minimize potential errors, make accurate measurements and create reproducible results.

For additional information about writing a procedure, see: Cookie Science 6: Baking it up.

Keep in mind that many experiments (even surveys, and certainly things like taste tests) require safety and consent paperwork before the experiments can begin. If you think you might be entering a science fair in the future, check the Society for Science website for all the rules and paperwork before you start any project!

For example, see Cookie Science 4: Cookie ethics.

Be sure to keep a detailed data table in your lab notebook throughout the duration of your project. Write down any relevant ideas you had or experiments that you did each day. Make sure to put the date on each page.

For more details about keeping a good lab notebook, see Cookie Science 3: The lab notebook.

V. Analyze and display your results.

In the final phase of a project, you should analyze your data and draw conclusions. When you analyze your data be sure to estimate your experimental error (the accuracy of your results) and the effects of any assumptions you made during the experiment. Perform other relevant statistical tests with your data to help determine if your hypothesis is true.

For more information on statistics, see:

Statistics: Make conclusions cautiously.

Cookie Science 8: The meaning of the mean.

Cookie Science 9: How data can spread.

Cookie Science 10: Finding the cookie difference.

Make tables and graphs of your experimental data. For all graphs and charts, it is important to include a title, labels for the axes and appropriate units for the variables. Wherever possible, use different colors to make it easier to identify different important aspects of the data. You should indicate experimental error on your graphs when possible, usually with small bars to show the standard deviation around the mean of each data point or bar height.

For additional information, see:

Cookie Science 14: One experiment, 400 cookies.

Cookie Science 11: That’s the way the cookie crumbles.

Cookie Science 15: Results aren’t always sweet.

Cookie Science 17: Posters – the good and the bad.

You should always aim to answer a standard set of questions in your results section. Below is a set of general questions that you should be able to answer after your analysis.

1. State your results. Was your hypothesis correct or incorrect? Explain.