By analyzing brain activity, a computer model can correctly guess which word a person is pondering, new research suggests. Eventually, the results may help scientists understand the roots of certain kinds of cognitive problems.

Reporting in the May 30 Science, a team used functional magnetic resonance imaging to track neural activity in volunteers shown pictures of airplanes, celery and 58 other everyday objects. Based on these brain scans, a computer model successfully sussed out which object and paired word people were observing — and therefore thinking about.

“The study is very clever; it's really an advance,” says Dedre Gentner, a cognitive psychologist at Northwestern University in Evanston, Ill., who was not involved in the study. Unlike other studies, this one watches the entire brain as it decodes language. “It doesn't assume that language processing goes on in a tiny square inch of brain,” she says.

The study, led by computer scientist Tom Mitchell of Carnegie Mellon University in Pittsburgh, played out like a guessing game. But the computer model first needed a rough idea of what each of the 60 different nouns used in the study “mean.” In order to home in on the meaning, the model searched for a given word — say, “airplane” —in a trillion words gathered from the Web, noting the key verbs neighboring the noun in question. “The word ‘airplane’ might frequently occur with the word ‘ride,’ but not frequently with ‘lick’ or ‘taste’,” says Mitchell. Knowing which verbs were often nearby “airplane,” the model could approach an approximate sense of the word's meaning.

After the computer model was trained, nine volunteers viewed 60 line drawings paired with words as functional MRI simultaneously recorded their brain activity.

Mitchell and his colleagues then trained the computer model to link 58 of those 60 nouns with the corresponding brain scans of the participants as they thought of the nouns. The trained computer then had to “guess” which brain scan matched each of the remaining two nouns. The researchers then repeated this partial training with a different subset of 58 words so that, ultimately, the computer had a try at guessing all of the 60 words. By chance, the correct word and brain scan should have been paired correctly half the time. The model got it right more than three out of four times, suggesting it could tell based on neural activity what word a person was pondering.

In some forms of dementia people forget specific words, like “poodle,” and can recall only the more general ones, like “dog” or “animal,” Mitchell says. Understanding how brains map meaning within context could help diagnose such diseases and help explain how people process language.

While the results are promising, knowing which words are often used together is not the same as understanding meaning, Gentner says, suggesting the approach may not work as well with parts of speech other than nouns. “If you try to apply this technique to verbs, it seems like it will be much harder to pull out their meaning,” she says. Opposites, like “give” and “take” or “answer” and “ask,” often appear in very similar contexts, but their meanings are radically different, she notes.

The research is still a far cry from true mind reading, adds Alfonso Caramazza, a neuropsychologist at Harvard University. “Reading brain scans and reading minds are two quite different things,” Caramazza says. “We need to have much better theories of the nature and structure of the mind before we can begin to think seriously about the task of reading minds.”

• Print
• |
• Comment