WaterSplatting Fast Underwater 3D Scene Reconstruction Using Gaussian Splatting

Huapeng Li University of Zurich
Wenxuan Song ETH Zurich
Tianao Xu ETH Zurich
Alexandre Elsig ETH Zurich
Jonas Kulhanek CTU in Prague,
ETH Zurich

WaterSplatting combines 3DGS with volume rendering to enable water/fog modeling

Abstract

We introduce WaterSplatting, a novel approach that fuses volumetric rendering with 3DGS to handle underwater data effectively. Our method employs 3DGS for explicit geometry representation and a separate volumetric field (queried once per pixel) for capturing the scattering medium. This dual representation further allows the restoration of the scenes by removing the scattering medium. Our method outperforms state-of-the-art NeRF-based methods in rendering quality on the underwater SeaThru-NeRF dataset. Furthermore, it does so while offering real-time rendering performance.

WaterSplatting overview
We start rendering by casting a ray per pixel and collect the patch-intersected Gaussians along the ray and their color given ray direction. Then, we walk through the list of sorted Gaussians per pixel, query their opacity and depth, based on which we acquire the transmittance of both Gaussians and medium, rendering the Gaussians and the segments between adjacent two to obtain the Medium component and the Object component.

Results

SeaThru-NeRF
FPS: 0.38
WaterSplatting
FPS: 27.4
Panama - comparison with SeaThru-NeRF Left:SeaThru-NeRF Right: WaterSplatting.
SeaThru-NeRF
FPS: 0.63
WaterSplatting
FPS: 59.4
Japanese Gradens - comparison with SeaThru-NeRF Left:SeaThru-NeRF Right: WaterSplatting.
WaterSplatting removed medium
IUI3 - w/o medium. Left:WaterSplatting reconstruction. Right: WaterSplatting with medium removed.
WaterSplatting removed medium Curasao - w/o medium. Left:WaterSplatting reconstruction. Right: WaterSplatting with medium removed.

Acknowledgements

This work was supported by the Czech Science Foundation (GAČR) EXPRO (grant no. 23-07973X), and by the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (ID:90254). Jonas Kulhanek acknowledges travel support from the European Union’s Horizon 2020 research and innovation programme under ELISE (grant no. 951847). The renderer is built on Splatfacto and SeaThru-NeRF (nerfstudio reimplementation). We thank all the authors for their great work and repos. Finally, we would also like to thank Dor Verbin for the video comparison tool used in this website.

Citation

Please use the following citation:
@article{li2024watersplatting,
  title={{W}ater{S}platting: Fast Underwater {3D} Scene Reconstruction using Gaussian Splatting},
  author={Li, Huapeng and Song, Wenxuan and Xu, Tianao and Elsig, Alexandre and Kulhanek, Jonas},
  journal={arXiv},
  year={2024}
}