Princeton engineers are working closely with neuroscientists to understand how visual information and words are encoded in the brain.
In a five-year collaboration, a team led by Princeton’s Peter Ramadge, chair and the Gordon Y.S. Wu Professor of electrical engineering, and James Haxby, a neuroscientist at Dartmouth College, have found common patterns in data from brain scans, called fMRI, that reveal brain activity as people perform tasks. The researchers are solving a long-standing challenge of comparing one person’s brain activity to another, which until now has been difficult because both the anatomy and functional processes of each person’s brain are different.
In one recent result, published in the journal Neuron, the researchers had subjects watch the entire movie “Raiders of the Lost Ark” while undergoing fMRI scans and used the data to derive a “common neural code” for how the brain recognizes complex visual images. Based on data from the first half of the movie, the researchers were able to predict, using only a person’s fMRI results, what scene he or she was watching in the second half of the movie.
Ramadge said the collaboration has not only revealed deep insights for neuroscience but has pushed the limits of the engineering techniques in ways that could be useful in many other areas. “It’s been a two-way street,” he said.
Ramadge attributed the success in part to weekly interdisciplinary meetings initiated by Jonathan Cohen, the Eugene Higgins Professor of Psychology and co-director of the Princeton Neuroscience Institute.
“It’s been a great way for my students and me to learn the language of neuroscience,” Ramadge said.
In a separate project, computer scientist David Blei and neuroscientist Ken Norman also are using fMRI data to understand how word meanings are represented in the brain and how these meanings shape memory retrieval. The researchers are showing how the meanings of words that were presented recently can linger in the brain and serve as a mental context that time-stamps memories, so that memories evoke words and vice versa. The work may aid the development of technologies for diagnosing and remediating memory problems.