This paper deals with the scarcity of data for training optical flow networks, highlighting the limitations of existing sources such as labeled synthetic datasets or unlabeled real videos. Specifically, we introduce a framework to generate accurate ground-truth optical flow annotations quickly and in large amounts from any readily available single real picture. Given an image, we use an off-the-shelf monocular depth estimation network to build a plausible point cloud for the observed scene. Then, we virtually move the camera in the reconstructed environment with known motion vectors and rotation angles, allowing us to synthesize both a novel view and the corresponding optical flow field connecting each pixel in the input image to the one in the new frame. When trained with our data, state-of-the-art optical flow networks achieve superior generalization to unseen real data compared to the same models trained either on annotated synthetic datasets or unlabeled videos, and better specialization if combined with synthetic images.

Learning optical flow from still images / Aleotti F.; Poggi M.; Mattoccia S.. - ELETTRONICO. - (2021), pp. 9578109.15196-9578109.15206. (Intervento presentato al convegno 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021 tenutosi a Nashville, TN, USA nel 20-25 June 2021) [10.1109/CVPR46437.2021.01495].

Learning optical flow from still images

Aleotti F.;Poggi M.;Mattoccia S.
2021

Abstract

This paper deals with the scarcity of data for training optical flow networks, highlighting the limitations of existing sources such as labeled synthetic datasets or unlabeled real videos. Specifically, we introduce a framework to generate accurate ground-truth optical flow annotations quickly and in large amounts from any readily available single real picture. Given an image, we use an off-the-shelf monocular depth estimation network to build a plausible point cloud for the observed scene. Then, we virtually move the camera in the reconstructed environment with known motion vectors and rotation angles, allowing us to synthesize both a novel view and the corresponding optical flow field connecting each pixel in the input image to the one in the new frame. When trained with our data, state-of-the-art optical flow networks achieve superior generalization to unseen real data compared to the same models trained either on annotated synthetic datasets or unlabeled videos, and better specialization if combined with synthetic images.
2021
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
15196
15206
Learning optical flow from still images / Aleotti F.; Poggi M.; Mattoccia S.. - ELETTRONICO. - (2021), pp. 9578109.15196-9578109.15206. (Intervento presentato al convegno 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021 tenutosi a Nashville, TN, USA nel 20-25 June 2021) [10.1109/CVPR46437.2021.01495].
Aleotti F.; Poggi M.; Mattoccia S.
File in questo prodotto:
File Dimensione Formato  
Aleotti_Learning_Optical_Flow_From_Still_Images_CVPR_2021_paper.pdf

accesso aperto

Tipo: Postprint
Licenza: Licenza per accesso libero gratuito
Dimensione 4.39 MB
Formato Adobe PDF
4.39 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/874906
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 21
  • ???jsp.display-item.citation.isi??? 5
social impact