Deep learning enables real-time imaging around corners

WASHINGTON — Researchers have harnessed the power of a type of artificial intelligence known as deep learning to create a new laser-based system that can image around corners in real time. With further development, the system might let self-driving cars “look” around parked cars or busy intersections to see hazards or pedestrians. It could also be installed on satellites and spacecraft for tasks such as capturing images inside a cave on an asteroid.

“Compared to other approaches, our non-line-of-sight imaging system provides uniquely high resolutions and imaging speeds,” said research team leader Christopher A. Metzler from Stanford University and Rice University. “These attributes enable applications that wouldn’t otherwise be possible, such as reading the license plate of a hidden car as it is driving or reading a badge worn by someone walking on the other side of a corner.”

In Optica, The Optical Society’s journal for high-impact research, Metzler and colleagues from Princeton University, Southern Methodist University, and Rice University report that the new system can distinguish submillimeter details of a hidden object from 1 meter away. The system is designed to image small objects at very high resolutions but can be combined with other imaging systems that produce low-resolution room-sized reconstructions.

“Non-line-of-sight imaging has important applications in medical imaging, navigation, robotics and defense,” said co-author Felix Heide from Princeton University. “Our work takes a step toward enabling its use in a variety of such applications.”

The new imaging system uses a commercially available camera sensor and a powerful, but otherwise standard, laser source that is similar to the one found in a laser pointer. The laser beam bounces off a visible wall onto the hidden object and then back onto the wall, creating an interference pattern known as a speckle pattern that encodes the shape of the hidden object.

Reconstructing the hidden object from the speckle pattern requires solving a challenging computational problem. Short exposure times are necessary for real-time imaging but produce too much noise for existing algorithms to work. To solve this problem, the researchers turned to deep learning.

“Compared to other approaches for non-line-of-sight imaging, our deep learning algorithm is far more robust to noise and thus can operate with much shorter exposure times,” said co-author Prasanna Rangarajan from Southern Methodist University. “By accurately characterizing the noise, we were able to synthesize data to train the algorithm to solve the reconstruction problem using deep learning without having to capture costly experimental training data.”

The researchers tested the new technique by reconstructing images of 1-centimeter-tall letters and numbers hidden behind a corner using an imaging setup about 1 meter from the wall. Using an exposure length of a quarter of a second, the approach produced reconstructions with a resolution of 300 microns.

The research is part of DARPA’s Revolutionary Enhancement of Visibility by Exploiting Active Light-fields (REVEAL) program, which is developing a variety of different techniques to image hidden objects around corners. The researchers are now working to make the system practical for more applications by extending the field of view so that it can reconstruct larger objects.

###

C. A. Metzler, F. Heide, P. Rangarajan, M. Madabhushi Balaji, A. Viswanath, A. Veeraraghavan, and R. G. Baraniuk,”Deep-inverse correlography: towards real-time high-resolution non-line-of-sight imaging,” 7 (1) 63-71 (2020).

DOI: https://doi.org/10.1364/OPTICA.374026.

Optica is an open-access, journal dedicated to the rapid dissemination of high-impact peer-reviewed research across the entire spectrum of optics and photonics. Published monthly by The Optical Society (OSA), Optica provides a forum for pioneering research to be swiftly accessed by the international community, whether that research is theoretical or experimental, fundamental or applied. Optica maintains a distinguished editorial board of more than 60 associate editors from around the world and is overseen by Editor-in-Chief Prem Kumar, Northwestern University, USA.

Founded in 1916, The Optical Society (OSA) is the leading professional organization for scientists, engineers, students and business leaders who fuel discoveries, shape real-life applications and accelerate achievements in the science of light. Through world-renowned publications, meetings and membership initiatives, OSA provides quality research, inspired interactions and dedicated resources for its extensive global network of optics and photonics experts. For more information, visit osa.org.

Aaron Cohen

(301) 633-6773

[email protected]

[email protected]