Computer model knows whether 'airplane' or 'celery' was on your mind
By analyzing brain activity, a computer model can correctly guess which word a person is pondering, new research suggests. Eventually, the results may help scientists understand the roots of certain kinds of cognitive problems.
Reporting in the May 30 Science, a team used functional magnetic resonance imaging to track neural activity in volunteers shown pictures of airplanes, celery and 58 other everyday objects. Based on these brain scans, a computer model successfully sussed out which object and paired word people were observing — and therefore thinking about.
“The study is very clever; it's really an advance,”
says Dedre Gentner, a cognitive psychologist at
The study, led by computer scientist Tom Mitchell of
After the computer model was trained, nine volunteers viewed 60 line drawings paired with words as functional MRI simultaneously recorded their brain activity.
Mitchell and his colleagues then trained the computer model to link 58 of those 60 nouns with the corresponding brain scans of the participants as they thought of the nouns. The trained computer then had to “guess” which brain scan matched each of the remaining two nouns. The researchers then repeated this partial training with a different subset of 58 words so that, ultimately, the computer had a try at guessing all of the 60 words. By chance, the correct word and brain scan should have been paired correctly half the time. The model got it right more than three out of four times, suggesting it could tell based on neural activity what word a person was pondering.
In some forms of dementia people forget specific words, like “poodle,” and can recall only the more general ones, like “dog” or “animal,” Mitchell says. Understanding how brains map meaning within context could help diagnose such diseases and help explain how people process language.
While the results are promising, knowing which words are often used together is not the same as understanding meaning, Gentner says, suggesting the approach may not work as well with parts of speech other than nouns. “If you try to apply this technique to verbs, it seems like it will be much harder to pull out their meaning,” she says. Opposites, like “give” and “take” or “answer” and “ask,” often appear in very similar contexts, but their meanings are radically different, she notes.
The research is still a far cry from true mind reading, adds
Alfonso Caramazza, a neuropsychologist at
Mitchell, T., S. Shinkareva, A.Carlson, K. Chang, V. Malave, R. Mason, M.Just.Predicting Human Brain Activity Associated with the Meanings of Nouns. Science 320, May 30,2008, 1161-1165.