NUS scientists develop a novel light-field sensor for 3D scene construction with unprecedented angular resolution

A research team from the National University of Singapore (NUS) Faculty of Science, led by Professor Liu Xiaogang from the Department of Chemistry, has developed a 3D imaging sensor that has an extremely high angular resolution, which is the capacity of an optical instrument to distinguish points of an object separated by a small angular distance, of 0.0018^o. This innovative sensor operates on a unique angle-to-colour conversion principle, allowing it to detect 3D light fields across the X-ray to visible light spectrum.

A light field encompasses the combined intensity and direction of light rays, which the human eyes can process to precisely detect the spatial relationship between objects. Traditional light sensing technologies, however, are less effective. Most cameras, for instance, can only produce two-dimensional images, which is adequate for regular photography but insufficient for more advanced applications, including virtual reality, self-driving cars, and biological imaging. These applications require precise 3D scene construction of a particular space.

For example, self-driving cars could use light-field sensing to view streets and more accurately assess road hazards so as to adjust their speed accordingly. Light-field sensing could also enable surgeons to accurately image a patient’s anatomy at varying depths, allowing them to make more precise incisions and better assess a patient’s risk of injury.

“Currently, light-field detectors use an array of lenses or photonic crystals to obtain multiple images of the same space from many different angles. However, integrating these elements into semiconductors for practical use is complicated and costly,” explained Prof Liu.  “Conventional technologies can detect light fields only in the ultraviolet to visible light wavelength range, leading to limited applicability in X-ray sensing.”

In addition, compared with other light-field sensors such as microlens arrays, the NUS team’s light field sensor has a larger angular measurement range of more than 80 degrees, high angular resolution which can potentially be less than 0.015 degrees for smaller sensors, and a wider spectral response range of between 0.002 nm and 550 nm. These specifications make the novel sensor able to capture 3D images at higher depth resolution.

The breakthrough was published in the prestigious journal Nature on 10 May 2023.

Made possible by perovskite nanocrystals

At the core of the novel light-field sensor are inorganic perovskite nanocrystals – compounds that have excellent optoelectronic properties. Due to their controllable nanostructures, perovskite nanocrystals are efficient light emitters, with an excitation spectrum that spans X-rays to visible light. The interactions between perovskite nanocrystals and light rays can also be tuned by carefully altering their chemical properties or by introducing small amounts of impurity atoms.

NUS researchers have patterned perovskite crystals onto a transparent thin-film substrate and integrated them into a colour charge-coupled device (CCD), which converts incoming light signals into a colour-coded output. This crystal-converter system comprises a basic functional unit of the light-field sensor.

When incident light hits the sensor, the nanocrystals become excited. In turn, the perovskite units emit their own light in varying colours depending on the angle at which the incoming light ray strikes. The CCD captures the emitted colour, which can then be used for 3D image reconstruction.

“A single angle value, however, is not enough to determine the absolute position of the object in a three-dimensional space,” shared by Dr Yi Luying, Research Fellow at the NUS Department of Chemistry and the first author of the paper. “We discovered that adding another basic crystal converter unit perpendicular to the first detector and combining it with a designed optical system could provide even more spatial information regarding the object in question.”

They then tested their light-field sensor in proof-of-concept experiments and found that their approach can indeed capture 3D images — with accurate reconstructions of depth and dimension — of objects placed 1.5 metres away.

Their experiments also demonstrated the capacity of the novel light-field sensor to resolve even very fine details. For example, a precise image of a computer keyboard was created that even captured the shallow protrusions of individual keys.

Future research

Prof Liu and his team are looking into methods to improve the spatial accuracy and resolution of their light-field sensor, such as using higher-end colour detectors. The team has also applied for an international patent for the technology.

“We will also explore more advanced technologies to pattern perovskite crystals more densely onto the transparent substrate, which could lead to better spatial resolution. Using materials other than perovskite may also expand the detection spectrum of the light-field sensor,” said Prof Liu.