When stressed, the brain goes ‘cheap’

When we’re not under pressure, we have time to reflect on the best course before making a decision. In times of stress, however, we fall back on quick and dirty decision making. A new study attempts to clarify how stress changes the way we perform working memory tasks.

Say you are thinking of getting a new car. You have a car but it’s getting old, and it’s time to move on. You might examine different models, check out the miles per gallon in hybrids, save up money and look at different financing methods.

Then your car breathes its last. You can’t get to work. You can hitch a ride for a few days, but you have to do something. Screw miles per gallon and financing. You run out and buy the car you can get the quickest, just like you did the last time this happened. It’s barely within your budget and it’s not as ecofriendly as you’d hoped. It’s probably not what you would have picked when you had ample time to consider. Is it the best decision?

That example shows two different kinds of decision making. Model-based decision making takes all the options into account. You examine the potential consequences of actions, look around at the environment and study all the possibilities. Model-free decision making is more “primitive” in style. Instead of carefully evaluating all the possible choices and outcomes, you just go with what worked best last time. Model-based decision making takes more focus and thought, while model-free decision making takes less.

Ross Otto and colleagues at New York University looked into how we make decisions. They gave 56 people a working memory test, and then put half of the volunteers’ hands up to the wrist into icy cold water, an unpleasant and physically stressful experience. The other half got room temperature water. After the stressor, the researchers took saliva from each participant to test for cortisol, a hormone indicative of stress. Then they sat the participants down in front of a computer screen for a two-step reinforcement learning task.

The task works like this: You see a black screen with two pictures. The one on the left leads to a green screen most of the time. The green screen has two pictures. The one on the right has a 60 percent chance of winning you a quarter, the one on the left, only 25 percent.  You go back to the black screen. If you pick the picture on the right, you have a higher chance of going to a blue screen. The two pictures on the blue screen also have different chances to earn a reward, but both are around 30 to 40 percent. But remember, you are dealing with uncertainty through all of this. The left choice on the black screen usually leads to a green screen, but sometimes it leads to the blue one.  In the blue and green screens, the pictures might get you a quarter, but you’re never certain you’ll get the money. How do you maximize your chances?

If you’re applying model-based learning, you would take time to learn what yields the most money as you performed the task. The right picture on the green screen has the highest chance of reward. OK. This means that if we want to make the most money, we need to get to the green screen the most often. So we need to take the choice on the left, which has the highest chance of getting to the green screen, and then pick the choice on the right, which has the highest chance of reward.

But if you are using model-free learning, you won’t think it through. Instead, you’re more likely to try the choices that got you a reward the last time. If the right choice on the blue screen worked, you’ll try that again, regardless of whether it’s really the most likely option.

In results published December 9 in the Proceedings of the National Academy of Sciences, Otto and his colleagues show that humans use a mixture of model-based and model-free learning. We lean toward the decisions that will make us the most money, but it’s hard to ignore the ones that made us money before, even if they aren’t the best option.

That’s how we might behave under normal circumstances. But what about after a few minutes of having your wrist stuck in ice water? What effect does stress have on how we make decisions?

The scientists showed that stress only hurts model-based learning. People who had been exposed to the ice water were less likely to try to work out the problem, and more likely to use what worked the first time. They also showed that participants who had a tough time with working memory (something important for model-based learning) were much more affected by the stress than those who got higher scores on working memory tests. This means that people who score well on working memory tests (a measure linked to general intelligence), are protected from the effects of the stressor. They can keep working out the model-based learning even under stress.

But what does this mean? Model-based learning is “more ‘expensive’ to carry out,” says Otto. “You have to think through what is happening. When there is stress, you’ll fall back on more primitive decision making.” So when people are in high stress situations, Otto says, “you may want to structure tasks so that model-free learning can be used.”

The results also assume that, most of the time, the model-based learning is the “better” option. But it is always better? There may be many situations where the mentally “cheaper” action is preferable. Heck, it may not even make a difference. In this test, people who relied on model-based learning did not end up taking home any more money than those who relied on model-free learning. They may have had higher working memory, but it didn’t make them earn more. Taking the test over a longer period of time might have shown more differences.

It also makes me wonder whether different stressors might have different effects. After all, cold water may not distract you that much. Would a social stressor like giving a speech have a bigger effect? Otto thinks that it might, but it will have to be tested.

Otto and colleagues hypothesize that differences in model-based and model-free learning under stress might have applications for things like understanding addiction. Addicts are most likely to relapse under times of stress. This is in part because stress induces things like drug craving. But Otto thinks that model-free learning might play a role. When recovering addicts are under stress, he says, they might be more likely to turn back to the things that rewarded them before.

It will be interesting to see whether model-free learning plays a role in drug relapse. And it would be interesting to see whether model-based learning is really “better” all the time. When faced with a simple choice, like which bathroom to use, I’m going with the one that was empty last time, no matter how urgent or stressful the situation. But when buying a car? Well, maybe I don’t want to wait until the stressful situation comes along. In some situations, it’s good to consider all the angles.