Sitemap

A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.

Pages

Page not found. Your pixels are in another canvas.

About me

projects

SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimation


Michaël Ramamonjisoa and Vincent Lepetit


@article{ramamonjisoa2019sharpnet, 
Title = {SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimation},
Author = {M. Ramamonjisoa and V. Lepetit},
Journal = {The IEEE International Conference on Computer Vision (ICCV) Workshops},
Year = {2019}
}

Published in International Conference on Computer Vision (ICCV) Workshop on 3D Reconstruction in the Wild, 2019

We introduce SharpNet, a method that predicts an accurate depth map for an input color image, with a particular attention to the reconstruction of occluding contours: Occluding contours are an important cue for object recognition, and for realistic integration of virtual objects in Augmented Reality, but they are also notoriously difficult to reconstruct accurately. For example, they are a challenge for stereo-based reconstruction methods, as points around an occluding contour are visible in only one image. Inspired by recent methods that introduce normal estimation to improve depth prediction, we introduce a novel term that constrains depth and occluding contours predictions. Since ground truth depth is difficult to obtain with pixel-perfect accuracy along occluding contours, we use synthetic images for training, followed by fine-tuning on real data. We demonstrate our approach on the challenging NYUv2-Depth dataset, and show that our method outperforms the state-of-the-art along occluding contours, while performing on par with the best recent methods for the rest of the images. Its accuracy along the occluding contours is actually better than the ‘‘ground truth’’ acquired by a depth camera based on structured light. We show this by introducing a new benchmark based on NYUv2-Depth for evaluating occluding contours in monocular reconstruction, which is our second contribution.

Predicting Sharp and Accurate Occlusion Boundaries in Monocular Depth Estimation Using Displacement Fields


Michaël Ramamonjisoa*, Yuming Du* and Vincent Lepetit
* Denotes equal contribution.


@article{ramamonjisoa2020dispnet, 
Title = {Predicting Sharp and Accurate Occlusion Boundaries in Monocular Depth Estimation Using Displacement Fields},
Author = {M. Ramamonjisoa and V. Lepetit},
Journal = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
Year = {2020}
}

Published in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020

Current methods for depth map prediction from monocular images tend to predict smooth, poorly localized contours for the occlusion boundaries in the input image. This is unfortunate as occlusion boundaries are important cues to recognize objects, and as we show, may lead to a way to discover new objects from scene reconstruction. To improve predicted depth maps, recent methods rely on various forms of filtering or predict an additive residual depth map to refine a first estimate. We instead learn to predict, given a depth map predicted by some reconstruction method, a 2D displacement field able to re-sample pixels around the occlusion boundaries into sharper reconstructions. Our method can be applied to the output of any depth estimation method and is fully differentiable, enabling end-to-end training. For evaluation, we manually annotated the occlusion boundaries in all the images in the test split of popular NYUv2-Depth dataset. We show that our approach improves the localization of occlusion boundaries for all state-of-the-art monocular depth estimation methods that we could evaluate, without degrading the depth accuracy for the rest of the images.

publications

On Object Symmetries and 6D Pose Estimation from Images


Giorgia Pitteri*, Michaël Ramamonjisoa*, Slobodan Ilic and Vincent Lepetit
* Denotes equal contribution.


Objects with symmetries are common in our daily life and in industrial contexts, but are often ignored in the recent literature on 6D pose estimation from images. In this paper, we study in an analytical way the link between the symmetries of a 3D object and its appearance in images. We explain why symmetrical objects can be a challenge when training machine learning algorithms that aim at estimating their 6D pose from images. We propose an efficient and simple solution that relies on the normalization of the pose rotation. Our approach is general and can be used with any 6D pose estimation algorithm. Moreover, our method is also beneficial for objects that are 'almost symmetrical', mph{i.e.} objects for which only a detail breaks the symmetry. We validate our approach within a Faster-RCNN framework on a synthetic dataset made with objects from the T-Less dataset, which exhibit various types of symmetries, as well as real sequences from T-Less.

@article{pitteri2019threedv, 
Title = {On Object Symmetries and 6D Pose Estimation from Images},
Author = {G. Pitteri and M. Ramamonjisoa and S. Ilic and V. Lepetit},
Journal = {International Conference on 3D Vision},
Year = {2019}
}

Published in 2019 International Conference on 3D Vision (3DV), 2019

SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimation


Michaël Ramamonjisoa and Vincent Lepetit


We introduce SharpNet, a method that predicts an accurate depth map for an input color image, with a particular attention to the reconstruction of occluding contours: Occluding contours are an important cue for object recognition, and for realistic integration of virtual objects in Augmented Reality, but they are also notoriously difficult to reconstruct accurately. For example, they are a challenge for stereo-based reconstruction methods, as points around an occluding contour are visible in only one image. Inspired by recent methods that introduce normal estimation to improve depth prediction, we introduce a novel term that constrains depth and occluding contours predictions. Since ground truth depth is difficult to obtain with pixel-perfect accuracy along occluding contours, we use synthetic images for training, followed by fine-tuning on real data. We demonstrate our approach on the challenging NYUv2-Depth dataset, and show that our method outperforms the state-of-the-art along occluding contours, while performing on par with the best recent methods for the rest of the images. Its accuracy along the occluding contours is actually better than the ''ground truth'' acquired by a depth camera based on structured light. We show this by introducing a new benchmark based on NYUv2-Depth for evaluating occluding contours in monocular reconstruction, which is our second contribution.

@article{ramamonjisoa2019sharpnet, 
Title = {SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimation},
Author = {M. Ramamonjisoa and V. Lepetit},
Journal = {The IEEE International Conference on Computer Vision (ICCV) Workshops},
Year = {2019}
}

Published in International Conference on Computer Vision (ICCV) Workshop on 3D Reconstruction in the Wild, 2019

Predicting Sharp and Accurate Occlusion Boundaries in Monocular Depth Estimation Using Displacement Fields


Michaël Ramamonjisoa*, Yuming Du* and Vincent Lepetit
* Denotes equal contribution.


Current methods for depth map prediction from monocular images tend to predict smooth, poorly localized contours for the occlusion boundaries in the input image. This is unfortunate as occlusion boundaries are important cues to recognize objects, and as we show, may lead to a way to discover new objects from scene reconstruction. To improve predicted depth maps, recent methods rely on various forms of filtering or predict an additive residual depth map to refine a first estimate. We instead learn to predict, given a depth map predicted by some reconstruction method, a 2D displacement field able to re-sample pixels around the occlusion boundaries into sharper reconstructions. Our method can be applied to the output of any depth estimation method and is fully differentiable, enabling end-to-end training. For evaluation, we manually annotated the occlusion boundaries in all the images in the test split of popular NYUv2-Depth dataset. We show that our approach improves the localization of occlusion boundaries for all state-of-the-art monocular depth estimation methods that we could evaluate, without degrading the depth accuracy for the rest of the images.

@article{ramamonjisoa2020dispnet, 
Title = {Predicting Sharp and Accurate Occlusion Boundaries in Monocular Depth Estimation Using Displacement Fields},
Author = {M. Ramamonjisoa and Y. Du and V. Lepetit},
Journal = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
Year = {2020}
}

Published in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020