Is what I see, what I imagine? Study finds neural overlap between vision and imagination

Medical University of South Carolina researchers report in Current Biology that the brain uses similar visual areas for mental imagery and vision, but it uses low-level visual areas less precisely with mental imagery than with vision.

These findings add knowledge to the field by refining methods to study mental imagery and vision. In the long-term, it could have applications for mental health disorders affecting mental imagery, such as post-traumatic stress disorder. One symptom of PTSD is intrusive visual reminders of a traumatic event. If the neural function behind these intrusive thoughts can be better understood, better treatments for PTSD could perhaps be developed.

The study was conducted by an MUSC research team led by Thomas P. Naselaris, Ph.D., associate professor in the Department of Neuroscience. The findings by the Naselaris team help answer an age-old question about the relationship between mental imagery and vision.

“We know mental imagery is in some ways very similar to vision, but it can’t be exactly identical,” explained Naselaris. “We wanted to know specifically in which ways it was different.”

To explore this question, the researchers used a form of artificial intelligence known as machine learning and insights from machine vision, which uses computers to view and process images.

“There’s this brain-like artificial system, a neural network, that synthesizes images,” Naselaris explained. “It’s like a biological network that synthesizes images.”

The Naselaris team trained this network to see images and then took the next step of having the computer imagine images. Each part of the network is like a group of neurons in the brain. Each level of the network or neuron has a different function in vision and then mental imagery.

To test the idea that these networks are similar to the function of the brain, the researchers performed an MRI study to see which brain areas are activated with mental imagery or vision.

While inside the MRI, participants viewed images on a screen and were also asked to imagine images at different points on the screen. MRI imaging enabled researchers to define which parts of the brain were active or quiet while participants viewed a combination of animate and inanimate objects.

Once these brain areas were mapped, the researchers compared the results from the computer model to human brain function.

They discovered that both the computer and human brains functioned similarly. Areas of the brain from the retina of the eye to the primary visual cortex and beyond are both activated with vision and mental imagery. However, in mental imagery, the activation of the brain from the eye to the visual cortex is less precise, and in a sense, diffuse. This is similar to the neural network. With computer vision, low-level areas that represent the retina and visual cortex have precise activation. With mental imagery, this precise activation become diffuse. In brain areas beyond the visual cortex, the activation of the brain or the neural network is similar for both vision and mental imagery. The difference lies in what’s happening in the brain from the retina to the visual cortex.

“When you’re imagining, brain activity is less precise,” said Naselaris. “It’s less tuned to the details, which means that the kind of fuzziness and blurriness that you experience in your mental imagery has some basis in brain activity.”

Naselaris hopes these findings and developments in computational neuroscience will lead to a better understanding of mental health issues.

The fuzzy dream-like state of imagery helps us to distinguish between our waking and dreaming moments. In people with PTSD, invasive images of traumatic events can become debilitating and feel like reality in the moment. By understanding how mental imagery works, scientists may better understand mental illnesses characterized by disruptions in mental imagery.

“When people have really invasive images of traumatic events, such as with PTSD, one way to think of it is mental imagery dysregulation,” explained Naselaris. “There’s some system in your brain that keeps you from generating really vivid images of traumatic things.”

A better understanding of how this works in PTSD could provide insight into other mental health problems characterized by mental imagery disruptions, such as schizophrenia.

“That’s very long term,” Naselaris clarified.

For now, Naselaris is focusing on how mental imagery works, and more research needs to be done to address the connection to mental health.

A limitation of the study is the ability to recreate fully the mental images conjured by participants during the experiment. The development of methods for translating brain activity into viewable pictures of mental images is ongoing.

This study not only explored the neurological basis of seen and imagined imagery but also set the stage for research into improving artificial intelligence.

“The extent to which the brain differs from what the machine is doing gives you some important clues about how brains and machines differ,” said Naselaris. “Ideally, they can point in a direction that could help make machine learning more brainlike.”

###

About the Medical University of South Carolina

Founded in 1824 in Charleston, MUSC is the oldest medical school in the South as well as the state’s only integrated academic health sciences center with a unique charge to serve the state through education, research and patient care. Each year, MUSC educates and trains more than 3,000 students and 800 residents in six colleges: Dental Medicine, Graduate Studies, Health Professions, Medicine, Nursing and Pharmacy. The state’s leader in obtaining biomedical research funds, in fiscal year 2019, MUSC set a new high, bringing in more than $284 million. For information on academic programs, visit http://musc.edu.

As the clinical health system of the Medical University of South Carolina, MUSC Health is dedicated to delivering the highest quality patient care available while training generations of competent, compassionate health care providers to serve the people of South Carolina and beyond. Comprising some 1,600 beds, more than 100 outreach sites, the MUSC College of Medicine, the physicians’ practice plan and nearly 275 telehealth locations, MUSC Health owns and operates eight hospitals situated in Charleston, Chester, Florence, Lancaster and Marion counties. In 2019, for the fifth consecutive year, U.S. News & World Report named MUSC Health the No. 1 hospital in South Carolina. To learn more about clinical patient services, visit http://muschealth.org.

MUSC and its affiliates have collective annual budgets of $3.2 billion. The more than 17,000 MUSC team members include world-class faculty, physicians, specialty providers and scientists who deliver groundbreaking education, research, technology and patient care.