Real-Time RGB-D Camera Pose Estimation in Novel Scenes using a Relocalisation Cascade

Camera relocalisation is a key computer vision problem. Common techniques match the current image against keyframes with known poses, directly regress the pose, or estimate the pose using camera-world correspondences. Regression forests have become popular for establishing correspondences. They are accurate, but previously needed to be trained offline on the target scene, preventing relocalisation in new environments. Recently, we showed how to adapt a pre-trained forest to a new scene online. The adapted forests perform similarly to offline forests, and can estimate the pose in near real time. Here, we extend this work to perform significantly better and run fully in real time: instead of simply accepting the pose produced by RANSAC, we score its final few hypotheses and select the best one; we chain several instances of our relocaliser (with different parameters) in a cascade, trying fast, less accurate relocalisation first, and falling back to slow, accurate relocalisation if needed; finally, we tune the hyperparameters of our relocalisers to achieve effective overall performance. We achieve state-of-the-art performance on the well-known 7-Scenes and Stanford 4 Scenes benchmarks. We visualise the internal behaviour of our forests, and show how to remove the need to train them offline on a generic scene.