Predicting Sharp and Accurate Occlusion Boundaries in Monocular Depth Estimation Using Displacement Fields

Michaël Ramamonjisoa^*, Yuming Du^* and Vincent Lepetit
^{* Denotes equal contribution.}

LIGM (UMR 8049) - Ecole des Ponts, UPE

CVPR 2020

[Paper]

[Code]

Abstract

Current methods for depth map prediction from monocular images tend to predict smooth, poorly localized contours for the occlusion boundaries in the input image. This is unfortunate as occlusion boundaries are important cues to recognize objects, and as we show, may lead to a way to discover new objects from scene reconstruction. To improve predicted depth maps, recent methods rely on various forms of filtering or predict an additive residual depth map to refine a first estimate. We instead learn to predict, given a depth map predicted by some reconstruction method, a 2D displacement field able to re-sample pixels around the occlusion boundaries into sharper reconstructions. Our method can be applied to the output of any depth estimation method and is fully differentiable, enabling end-to-end training. For evaluation, we manually annotated the occlusion boundaries in all the images in the test split of popular NYUv2-Depth dataset. We show that our approach improves the localization of occlusion boundaries for all state-of-the-art monocular depth estimation methods that we could evaluate, without degrading the depth accuracy for the rest of the images.

Method

[GitHub]

Paper and Supplementary Material

Michaël Ramamonjisoa^*, Yuming Du^* and Vincent Lepetit
^{* Denotes equal contribution.}
Predicting Sharp and Accurate Occlusion Boundaries in Monocular Depth Estimation Using Displacement Fields
In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Poster

Video

Dataset

Access our NYUv2OC++ dataset here

Citation

Please consider citing our paper if you found it helpful for your projects:

@InProceedings{Ramamonjisoa_2020_CVPR, 
 author = {Ramamonjisoa, Michael and Du, Yuming and Lepetit, Vincent}, 
 title = {Predicting Sharp and Accurate Occlusion Boundaries in Monocular Depth Estimation Using Displacement Fields}, 
 booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
 month = {June},
 year = {2020}
 }

Acknowledgements

This webpage template was borrowed from some colorful folks.