Speaker: Jakub Nawrocki
Abstract
In this thesis we investigate the topic of surface reconstruction from multi-view RGB images using Gaussian Splatting. Since Gaussian Splatting is fundamentally a novel view synthesis method, the geometric information stored in the trained model, is not sufficient for accurate surface reconstruction. Recent works solve this by using a large variety of geometric regularizations and joint optimization of geometric information with the rendered views. Additionally, image datasets can contain features that are difficult to represent using the Gaussian model, such as transparency, reflections, metallic surfaces and flat mono-colored surfaces. These difficult features, contribute to a local increase in noise. In order to address both the inherent, and feature specific noise, we propose a surface reconstruction pipeline, based on Relightable3DGaussian (R3DG). First, we introduce a stochastic differentiable Poisson surface reconstruction (SDPSR) solver, which lets us efficiently extract a signed distance field (SDF), during the Gaussian Splatting training process. The SDF is then used to add geometric regularization in the Gaussian Splatting network. Next, we use the probabilities obtained from the SDPSR solver, to influence the Gaussian Splat spawning process and encourage a more even distribution along the geometric surface. Finally, we observe that R3DG produces view-inconsistent depth and normal renders, hindering optimization of our geometric regularization terms. To fix this, we introduce additional regularizations, aimed increasing depth consistency. We evaluate our solution, on the DTU dataset. We also create a small synthetic dataset, to test the effect of our method on specific, difficult-to-reconstruct features.
