Joint Graph-based Depth Refinement and Normal EstimationView Publication
Depth estimation is an essential component in understanding the 3D geometry of a scene, with numerous applications in urban and indoor settings. These scenarios are characterized by a prevalence of human made structures, which in most of the cases are either inherently piece-wise planar or can be approximated as such. With these settings in mind, we devise a novel depth refinement framework that aims at recovering the underlying piece-wise planarity of those inverse depth maps associated to piece-wise planar scenes. We formulate this task as an optimization problem involving a data fidelity term, which minimizes the distance to the noisy and possibly incomplete input inverse depth map, as well as a regularization, which enforces a piece-wise planar solution. As for the regularization term, we model the inverse depth map pixels as the nodes of a weighted graph, with the weight of the edge between two pixels capturing the likelihood that they belong to the same plane in the scene. The proposed regularization fits a plane at each pixel automatically, avoiding any a priori estimation of the scene planes, and enforces that strongly connected pixels are assigned to the same plane. The resulting optimization problem is solved efficiently with the ADAM solver. Extensive tests show that our method leads to a significant improvement in depth refinement, both visually and numerically, with respect to state-of-the-art algorithms on the Middlebury, KITTI and ETH3D multi-view datasets.
Related PublicationsView All
BP-MVSNet: Belief-Propagation-Layers for Multi-View-Stereo
Christian Sormann*, Patrick Knobelreiter*, Andreas Kuhn, Mattia Rossi, Thomas Pock*, Friedrich Fraundorfer*In this work, we propose BP-MVSNet, a convolutional neural network (CNN)-based Multi-View-Stereo (MVS) method […]
DeepC-MVS: Deep Confidence Prediction for Multi-View Stereo Reconstruction
Andreas Kuhn, Christian Sormann*, Mattia Rossi, Oliver Erdler, Friedrich Fraundorfer*Deep Neural Networks (DNNs) have the potential to improve the quality of image-based 3D reconstructions. Howev […]