Imagination is not just replaying what we see and hear

If you were to imagine a waterfall, a misty cascade into an azure pool surrounded by towering trees might come to mind. That mental vision might also be accompanied by the imagined roar of water splashing down. But when it comes to our brains, does imagining a waterfall activate different areas compared with seeing or hearing one in real life?

For both sounds and sights, the overlap between imagination and perception appears not in brain areas linked to a single sense, but in high-level areas that accept multiple types of sensory inputs, researchers report March 31 in Neuron.

For years, cognitive neuroscientist Rodrigo Braga has been working to determine whether the human brain is processing mental imagery through hearing and other senses or whether something else is at play.

“When I was a teenager, I remember the first time realizing that there’s like a voice I can hear in my head and thinking, ‘Oh, that’s really strange’,” says Braga, of Northwestern University Feinberg School of Medicine in Chicago. 

In this study, he and his colleagues prompted eight participants to imagine scenes, faces, someone else speaking, internal monologues and sounds while in an MRI scanner. The small number of individuals allowed the researchers to collect hours of MRI data to create individualized brain maps rather than averaging across individuals. This technique allowed the team to reliably find individual variation in brain activity during imagination.

The prompts they used were open-ended, like “imagine a castle on a hill” or “imagine a rock song playing on the radio.” After each prompt, they asked participants about both the visual and auditory experience. The main aspect of experience that they focused on in the scanner was vividness, or how clear and realistic the experience felt.

Outside of the scanner, the team then asked follow-up questions to understand the details of each experience and what aspects made one mental image more vivid than another. For instance, they asked participants how much they agreed or disagreed with statements such as “I envisioned the location of objects, people or places.”

“This is one of the many papers that will come out in the next years that try to break down this obscure concept of vividness,” says cognitive psychologist Alfredo Spagna of John Cabot University in Rome, who was not involved in the study.

The team used this break down of vividness to group their data into two buckets: locations and events on the one hand and speech and language on the other. In trials when participants reported thinking about locations or events, they reported high visual vividness. Activity also went up in their brains’ “default network A,” a system linked to spatial processing. When thinking about speech or language, participants reported high auditory vividness and engaged the language network, which is usually involved in reading or listening to speech.

Both networks are what the researchers call “transmodal,” meaning they respond to new information regardless of the sense it came through.

While other studies have observed activity in visual sensory or representation areas in the brain while participants imagine specific objects they’ve seen recently, this study’s holistic prompts yielded different results. Basic visual sensory areas respond to details like edges, colors and line orientations, says cognitive neuroscientist Nathan Anderson of Brigham Young University in Provo, Utah. “There’s some evidence that people don’t necessarily imagine fine details when they are imagining a holistic scene.”

The results aren’t particularly surprising given that the prompts didn’t ask participants to imagine detailed visual or auditory experiences, says Stephen Kosslyn, a cognitive neuroscientist at Harvard University, who was not involved in this study. “It would be useful to try to pull apart the contributions of the various components of their complex tasks.”

Spagna, for one, sees these open-ended prompts as a strength of the study. Mental imagery is probably closer to imagining a castle on a hill than it is to having to imagine minute details of an image you saw on a screen, he says. “This is what imagery is for.”