A new way for machines to process visual information based on human sight

PITTSBURGH (June 20, 2019) — Self-driving cars rely on their ability to accurately “see” the road ahead and make adjustments based on what they see. They need to, for instance, react to a pedestrian who steps out from between parked cars, or know to not turn down a road that is unexpectedly closed for construction. As such technology becomes more ubiquitous, there’s a growing need for a better, more efficient way for machines to process visual information.

New research from the University of Pittsburgh will develop a neuromorphic vision system that takes a new approach to capturing visual information that is based on the human brain, benefitting everything from self-driving vehicles to neural prosthetics.

Ryad Benosman, PhD, professor of ophthalmology at the University of Pittsburgh School of Medicine who holds appointments in electrical engineering and bioengineering, and Feng Xiong, PhD, assistant professor of electrical and computer engineering at the Swanson School of Engineering, received $500,000 from the National Science Foundation (NSF) to conduct this research.

Conventional image sensors record information frame-by-frame, which stores a great deal of redundant data along with that which is useful. This excess data storage occurs because most pixels do not change from frame to frame, like stationary buildings in the background. Inspired by the human brain, the team will develop a neuromorphic vision system driven by the timings of changes in the dynamics of the input signal, instead of the conventional image-based system.

“With existing neuromorphic camera systems, the communication between the camera and the computing system is limited by how much data it is trying to push through, which negates the benefits of the large bandwidth and low power consumption that this camera provides,” says Dr. Xiong. “We will use a spiking neural network with realistic dynamic synapses that will enhance computational abilities, develop brain-inspired machine learning to understand the input, and connect it to a neuromorphic event-based silicon retina for real-time operating vision.”

This system will work more efficiently than existing technology, with orders of magnitude better energy efficiency and bandwidth.

“We believe this work will lead to transformative advances in bio-inspired neuromorphic processing architectures, sensing, with major applications in self-driving vehicles, neural prosthetics, robotics and general artificial intelligence,” says Dr. Benosman.

The grant will begin July 1, 2019, and is expected to last until June 30, 2022.

###

About the Swanson School of Engineering

The University of Pittsburgh’s Swanson School of Engineering is one of the oldest engineering programs in the U.S. and is consistently ranked among the top 25 public engineering programs by U.S. News & World Report. The Swanson School has excelled in basic and applied research during the past decade with focus areas in sustainability, energy systems, advanced manufacturing, bioengineering, micro- and nano-systems, computational modeling and advanced materials development.

About the University of Pittsburgh School of Medicine

As one of the nation’s leading academic centers for biomedical research, the University of Pittsburgh School of Medicine integrates advanced technology with basic science across a broad range of disciplines in a continuous quest to harness the power of new knowledge and improve the human condition. Driven mainly by the School of Medicine and its affiliates, Pitt has ranked among the top 10 recipients of funding from the National Institutes of Health since 1998. In rankings recently released by the National Science Foundation, Pitt ranked fifth among all American universities in total federal science and engineering research and development support.

Likewise, the School of Medicine is equally committed to advancing the quality and strength of its medical and graduate education programs, for which it is recognized as an innovative leader, and to training highly skilled, compassionate clinicians and creative scientists well-equipped to engage in world-class research. The School of Medicine is the academic partner of UPMC, which has collaborated with the University to raise the standard of medical excellence in Pittsburgh and to position health care as a driving force behind the region’s economy. For more information about the School of Medicine, see http://www.medschool.pitt.edu.