Document detail
ID
oai:arXiv.org:2408.04831
Topic
Computer Science - Computer Vision... Computer Science - Artificial Inte...Author
Du, Bi'an Meng, Lingbei Hu, WeiCategory
Computer Science
Year
2024
listing date
1/8/2025
Keywords
sparse inputs model images computer 3dAbstract
Sparse-view 3D reconstruction is a major challenge in computer vision, aiming to create complete three-dimensional models from limited viewing angles.
Key obstacles include: 1) a small number of input images with inconsistent information; 2) dependence on input image quality; and 3) large model parameter sizes.
To tackle these issues, we propose a self-augmented two-stage Gaussian splatting framework enhanced with structural masks for sparse-view 3D reconstruction.
Initially, our method generates a basic 3D Gaussian representation from sparse inputs and renders multi-view images.
We then fine-tune a pre-trained 2D diffusion model to enhance these images, using them as augmented data to further optimize the 3D Gaussians.
Additionally, a structural masking strategy during training enhances the model's robustness to sparse inputs and noise.
Experiments on benchmarks like MipNeRF360, OmniObject3D, and OpenIllumination demonstrate that our approach achieves state-of-the-art performance in perceptual quality and multi-view consistency with sparse inputs.
Du, Bi'an,Meng, Lingbei,Hu, Wei, 2024, AugGS: Self-augmented Gaussians with Structural Masks for Sparse-view 3D Reconstruction
