Recent advancements have shown the potential of leveraging both point clouds and images to localize anomalies. Nevertheless, their applicability in industrial manufacturing is often constrained by significant drawbacks, such as the use of memory banks, which lead to a substantial increase in terms of memory footprint and inference time. We propose a novel light and fast framework that learns to map features from one modality to the other on nominal samples and detect anomalies by pinpointing inconsistencies between observed and mapped features. Extensive experiments show that our approach achieves state-of-the-art detection and segmentation performance, in both the standard and few-shot settings, on the MVTec 3D-AD dataset while achieving faster inference and occupying less memory than previous multimodal AD methods. Furthermore, we propose a layer pruning technique to improve memory and time efficiency with a marginal sacrifice in performance.

Costanzino, A., ZAMA RAMIREZ, P., Lisanti, G., DI STEFANO, L. (2024). Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping [10.1109/CVPR52733.2024.01631].

Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping

Alex Costanzino;Pierluigi Zama Ramirez;Giuseppe Lisanti;Luigi Di Stefano
2024

Abstract

Recent advancements have shown the potential of leveraging both point clouds and images to localize anomalies. Nevertheless, their applicability in industrial manufacturing is often constrained by significant drawbacks, such as the use of memory banks, which lead to a substantial increase in terms of memory footprint and inference time. We propose a novel light and fast framework that learns to map features from one modality to the other on nominal samples and detect anomalies by pinpointing inconsistencies between observed and mapped features. Extensive experiments show that our approach achieves state-of-the-art detection and segmentation performance, in both the standard and few-shot settings, on the MVTec 3D-AD dataset while achieving faster inference and occupying less memory than previous multimodal AD methods. Furthermore, we propose a layer pruning technique to improve memory and time efficiency with a marginal sacrifice in performance.
2024
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
17234
17243
Costanzino, A., ZAMA RAMIREZ, P., Lisanti, G., DI STEFANO, L. (2024). Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping [10.1109/CVPR52733.2024.01631].
Costanzino, Alex; ZAMA RAMIREZ, Pierluigi; Lisanti, Giuseppe; DI STEFANO, Luigi
File in questo prodotto:
File Dimensione Formato  
Costanzino_Multimodal_Industrial_Anomaly_Detection_by_Crossmodal_Feature_Mapping_CVPR_2024_paper.pdf

accesso aperto

Tipo: Postprint
Licenza: Licenza per accesso libero gratuito
Dimensione 6.33 MB
Formato Adobe PDF
6.33 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/968592
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 6
  • ???jsp.display-item.citation.isi??? ND
social impact