Generative Inpainting of large missing areas in Point Clouds

Hole Filling is an important task when it comes to reconstructing point clouds or surfaces with missing data. Scanning devices like LiDAR capture scenes in real life. In general, such scans consist of point clouds with depth data embedded as position. One key problem that arises is that objects in the scene can rarely be captured completely, the result is missing information of the objects. Previous hole filling methods rely on pre-trained models which mostly consist of 3D data. While such methods handle In-domain objects relatively well, they lack the ability to generate objects which are not well represented in the dataset they are trained on. In order to alleviate these issues, hole filling techniques begin to shift towards a generative nature. This thesis aims to extend existing 2D generative models to create a method that can reconstruct point clouds with large missing areas.

Bildquelle: Yoni Kasten, Ohad Rahamim, and Gal Chechik. Point-cloud completion with pretrained text-to-image diffusion models, 2023.