From Dark Flash Images to Relightable 3D Scenes with Photometric Stereo Priors
Xuening Zhu , Renjiao Yi , Xiaohong Chen , Xin Wen , Xuesong Xu , Hailiang Hou , Kai Xu , Chenyang Zhu
Radiance fields, such as NeRFs, 3D Gaussians, and their variants, have emerged as the leading representations for 3D scene reconstruction due to their exceptional performance in novel view synthesis. However, their effectiveness depends on input images captured in well-lit, static environments, making dark scenes a significantly challenging case. Prior works employ low-light enhancement for low-light scenes (e.g., candlelight), but completely dark scenes remain an unsolved problem. However, this is a very common case when exploring unknown scenes, such as caves or nighttime forests, or derelict buildings. To solve the problem, we propose capturing images with a camera-mounted flashlight for exploring such scenes, which is an easily accessible setting for robots. The flashlight’s parameters are modeled and optimized in the reconstruction pipeline, including the flashlight’s angular and distance attenuation, position, rotation, and intensities. Under this setting, the captured images are under dynamic lighting conditions, i.e., lighting is changing for each image. We formulate a photometric stereo (PS) problem of input images by a grouping-and-merging strategy, leveraging its results as supervision priors. As a result, the method enables reconstruction and relighting of dark scenes. Experiments show that the method outperforms state-of-the-art approaches in decomposition, geometry, and relighting.
3D Reconstruction / Dark-scene Reconstruction / Image Relighting
Higher Education Press
/
| 〈 |
|
〉 |
PDF
(22402KB)
