Researchers Reconstruct Relightable 3D Scenes from Dark Flash Images

Researchers have developed Dark-2DGS, a novel framework that combines photometric stereo priors with 2D Gaussian Splatting to reconstruct high-fidelity, relightable 3D scenes from flash photography captured in dark environments. In an experimental study published in Frontiers of Computer Science, the method demonstrated higher geometric accuracy and visual quality than existing neural rendering approaches when tested on both synthetic datasets and real-world dark scenes.

High-quality 3D content reconstruction has long been a primary focus of research in computer vision and graphics. Reconstructing scenes in dark environments presents a difficult challenge because flash photography, which serves as the primary light source, introduces complex photometric variations. Existing neural rendering methods typically suffer from geometric noise and a loss of fine details when attempting to process such low-light data.

To overcome these limitations, a research team from the National University of Defense Technology and Hunan Province’s Xiangjiang Laboratory designed the Dark-2DGS framework. The research group included Academician Chen Xiaohong, Professor Xu Kai, Associate Professor Yi Renjiao, and Professor Zhu Chenyang, among others, with their study appearing in the journal co-published by Higher Education Press and Springer Nature.

In developing the framework, the researchers analyzed photometric cues in flash images and employed a physical lighting model to guide the geometry optimization of the neural field. To reconstruct 3D scenes from dark data with greater accuracy, the authors designed a coarse-to-fine optimization strategy that integrates surface material information directly into the rendering pipeline to detect and recover intricate details.

The reconstruction pipeline begins by using standard 2D Gaussian Splatting to establish the initial geometry of the scene. Following this step, the system introduces a Photometric Stereo enhancement module that analyzes illumination variations across multi-view images to estimate surface feature values, including material properties, which are then integrated into the optimization process to refine both the geometry and the material.

Finally, the method performs inverse rendering to decompose scene attributes. Experimental data demonstrated that, compared with existing view synthesis methods, the proposed approach is more robust to low-light conditions, provides high reconstruction accuracy, and successfully supports relighting.

Subject of Research: 3D scene reconstruction and relighting from dark flash images using photometric stereo priors

Article Title: From dark flash images to relightable 3D scenes with photometric stereo priors

Article References: From dark flash images to relightable 3D scenes with photometric stereo priors. Original publication (Bibliographic details unavailable.)

Image Credits: AI Generated

DOI: Not provided

Keywords: 3D reconstruction, neural rendering, 2D Gaussian Splatting, photometric stereo, flash photography, computer vision, inverse rendering, relighting, dark, flash, images, relightable