LadyBird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D Reconstruction with Symmetry

Yifan Xu1, Tianqi Fan2,3, Yi Yuan1, Gurprit Singh3
1Netease Fuxi AI Lab, Hangzhou, China, 2Saarland Informatics Campus, Saarbrücken, Germany, 3Max Planck Institute for Informatics, Saarbrücken, Germany
ECCV 2020 (Oral)
We study the effect of pointset discrepancy on network training and propose FPS-based sampling approach that theoretically encourages better generalization performance. This results in fast convergence for SGD-based optimization algorithms.


Deep implicit field regression methods are effective for 3D reconstruction from single-view images. However, the impact of different sampling patterns on the reconstruction quality is not well-understood. In this work, we first study the effect of point set discrepancy on the network training. Based on Farthest Point Sampling algorithm, we propose a sampling scheme that theoretically encourages better generalization performance, and results in fast convergence for SGD-based optimization algorithms. Secondly, based on the reflective symmetry of an object, we propose a feature fusion method that alleviates issues due to selfocclusions which makes it difficult to utilize local image features. Our proposed system Ladybird is able to create high quality 3D object reconstructions from a single input image. We evaluate Ladybird on a large scale 3D dataset (ShapeNet) demonstrating highly competitive results in terms of Chamfer distance, Earth Mover’s distance and Intersection Over Union (IoU).




Thanks to all the anonymous reviewers for their comments.

Imprint / Data Protection