How the brain integrates sensory input

Researchers from Bielefeld University, Oxford University (Great Britain), and Aix-Marseille University (France) investigated this phenomenon of flexibility in perception, and have now published a study on their findings that appears in the scientific journal Neuron (29 April 2019). In their publication, the researchers reveal where sensory stimuli are integrated in the brain, and in which area of the brain this flexibility can be located. From Bielefeld University, Professor Dr. Christoph Kayser and Dr. Hame Park from the Cluster of Excellence CITEC were involved in the study.

“We are interested in how the brain processes sensory input,” says Kayser, who heads the “Cognitive Neuroscience” research group. In his work, Kayser deals with multi-sensory integration – the combination of various sensory data. This happens, for instance, when watching a movie: you hear what the characters are saying to each other while at the same time watching the movements of their lips. It is not always useful, however, for auditory and visual information to be automatically integrated in the brain: one example of this would be watching a foreign-language film that is dubbed and the movements of the actors’ lips do not match the spoken sounds.

The researchers’ study was looking to identify the areas of the brain in which flexible sensory integration takes place. For this, they tested out three potential models. While different sensory stimuli were processed completely apart from each other in the first model, they were automatically integrated in the second model. The third variant was the model of “causal inference” in which different sensory stimuli are only integrated if they are not distant from each other in spatial or chronological terms. For example, if you always hear a sound and see an image at the same time, the brain integrates this information. However, if the sound and image appear together, they will not be integrated – even though they were previously separate from each other. “In the “causal inference” model, the brain infers that the source of the sensory stimuli might be the same. Sensory stimuli are therefore not automatically integrated – this is only the case if they do originate from the same source,” says Kayser.

To test these three models, study participants were exposed to visual and auditory stimuli. In the experiment, light and sound stimuli were shown, at times simultaneously, and at other times with different frequencies. Meanwhile, the researchers recorded the brain activity of their study participants using magnetoencephalography (MEG). They found that the three models match up to different areas in the brain, and thus correspond with distinct levels of processing. At the lowest level, sensory information is mapped separately in the visual and auditory cortexes. Following this, this information is automatically integrated in the parietal lobe, which is located in the upper area of the brain. Only at a higher level of processing does the brain parse out the information from the previous stages and, if necessary, filter out disruptive sensory stimuli. This flexibility in perception is located in special areas of the frontal lobe that are responsible for abstract thinking. “It has long been known how people deal with different sensory information – but at the level of behavior. With our study, we were able to show for the first time how and where in the brain this kind of information is processed,” says Kayser.

The findings of the study could be of use in many other areas of research, particularly in work on abstract thinking, as flexibility and causal relationships play an important role in this process. “How the brain processes sensory input is relevant for technical applications, including the integration of humans and machines,” says Kayser. This is a key topic for Kayser’s colleagues at Bielefeld University’s Cluster of Excellence CITEC. The findings of this study are also important in the clinical context, where the findings could be used to help better understand diseases in which patients have difficulty in correctly processing sensory information, such as autism.

Kayser is a biologist and mathematician, and in 2017, he accepted a professorship in cognitive neuroscience at Bielefeld University. In 2015, he was awarded a “Consolidator Grant” from the European Research Council, which runs until 2020. With this award, the European Research Council supports promising researchers at the beginning of their independent careers.

###