Unoriented 3D Point Cloud Reconstruction by Optimising Differential Fields

Abstract

Reconstructing 3D objects from sensor data, such as point clouds, is a fundamental problem in computer vision and graphics, with applications in augmented and virtual reality, robotics, medical imaging, and industrial inspection. While numerous techniques exist, implicit field-based methods have gained significant traction due to their flexibility and ability to model complex shapes. Recent advances in differentiable fields, particularly fields modelled by neural networks, have further improved reconstruction quality by leveraging deep learning and gradient-based optimisation.

This thesis focuses on the challenging task of surface reconstruction of watertight shapes, characterised by distinct interior and exterior regions, from unoriented point clouds, which do not have normals. Without orientation information, determining a consistent inside-outside partitioning becomes difficult, and so optimisers typically find poor local optima. We address this issue through four major contributions. The first three contributions relate to improving neural signed distance function (SDF) approaches for unoriented surface reconstruction. First, we propose a divergence-based regularisation with an annealing schedule to encourage smooth and reliable gradient orientations, guiding optimisation toward more favourable minima. Second, we introduce a novel energy function over volume partitions for discrete surface reconstruction, optimised via a move-making algorithm. This discrete solution is then used to inform the neural SDF, improving coarse orientation and reducing the risk of poor local minima. Third, we develop a homotopy-based optimisation strategy that incrementally deforms an initially simple problem into the full reconstruction task, stabilising the optimisation path and preventing premature convergence to suboptimal solutions. In our final contribution, we explore the recent improvements for the task with generalised winding number optimisation, and analyse their benefits over neural SDFs. We then improve upon the existing approximation of winding numbers for point clouds, deriving a better approximation and showing that this improves two distinct approaches for winding number optimisation.

Beyond surface reconstruction, we further contribute to the broader study of implicit neural representations (INRs), a class of neural networks that includes neural SDFs. Specifically, we improve the trade-off between representation capacity and parameter size using a mixture-of-experts framework, allowing specialised expert regions to be optimised during reconstruction. Additionally, we introduce a general initialisation strategy that applies to any activation function, unifying and improving upon existing initialisation methods. We demonstrate that this is particularly valuable for INRs, where non-standard activation functions with vastly different properties are commonly used.

These contributions enhance the robustness and efficiency of differentiable field-based surface reconstruction, improving both implicit neural representations and optimisation techniques. Our findings provide new insights into overcoming the challenges of unoriented reconstruction and contribute to the broader field of 3D computer vision. Show more