Learning from scratch a confidence measure

Stereo vision is a popular technique to infer depth from two or more images. In this field, confidence measures, typically obtained from the analysis of the cost volume, aim at detecting uncertain disparity assignments. As recently proved, multiple confidence measures combined with hand-crafted features extracted from the cost volume can be used also for other purposes and in particular to improve the overall disparity accuracy leveraging on machine learning techniques. In this paper, starting from the observation that recurrent local patterns occurring in the disparity maps can tell a correct assignment from a wrong one, we follow a completely different methodology to infer a novel confidence measure from scratch. Specifically, leveraging on Convolutional Neural Networks, we pose the confidence formulation as a regression problem by analyzing the disparity map provided by a stereo vision system. Once trained on a subset of the KITTI 2012 dataset with the disparity maps provided by the simple block-matching algorithm, our confidence measure outperforms state-of-the-art with two datasets (KITTI 2015 and Middlebury 2014) as well as with two stereo algorithms. The experimental evaluation reported clearly highlights that our approach is capable to better generalize its behavior in different circumstances with respect to state-of-the-art. Finally, not being based on cost volume analysis, our proposal is also potentially suited for out-of-the-box depth generation devices which usually do not expose the cues required by top-performing approaches.