VS-Sim: A Synthetic Dataset for Viewpoint Shift Robustness

We release VS-Sim, a synthetic dataset of road scene images that can be used to study the robustness to viewpoint shift of Computer Vision models for several tasks. Our dataset includes images from a frontal camera in different positions, and annotations for tasks both in frontal view (depth, semantic segmentation) and Bird’s Eye View (semantic segmentation). The paper also includes an analysis of the robustness of a Bird’s Eye View segmentation model to viewpoint shifts. Experiments indicate that viewpoint shift significantly degrades the performance of a model trained on data from a single viewpoint, and suggest that using multiple viewpoints during training helps to mitigate the performance drop across different scenarios.