Existing Continual Learning (CL) solutions only partially address the constraints on power, memory and computation of the deep learning models when deployed on low-power embedded CPUs. In this paper, we propose a CL solution that embraces the recent advancements in CL eld and the e ciency of the Binary Neural Networks (BNN), that use 1-bit for weights and activations to e ciently execute deep learning models. We propose a hybrid quantization of CWR* (an e ective CL approach) that considers di erently forward and backward pass in order to retain more precision during gradient update step and at the same time minimizing the latency overhead. The choice of a binary network as backbone is essential to meet the constraints of low power devices and, to the best of authors' knowledge, this is the rst attempt to prove on-device learning with BNN. The experimental validation carried out con rms the validity and the suitability of the proposed method.

On-Device Learning with Binary Neural Networks / Lorenzo Vorabbi, Davide Maltoni, Stefano Santi. - STAMPA. - 14365:(2024), pp. 39-50. (Intervento presentato al convegno Image Analysis and Processing - ICIAP 2023 Workshops. ICIAP 2023 tenutosi a Udine nel 09/2023) [10.1007/978-3-031-51023-6_4].

On-Device Learning with Binary Neural Networks

Lorenzo Vorabbi
Primo
;
Davide Maltoni
Secondo
;
Stefano Santi
Ultimo
2024

Abstract

Existing Continual Learning (CL) solutions only partially address the constraints on power, memory and computation of the deep learning models when deployed on low-power embedded CPUs. In this paper, we propose a CL solution that embraces the recent advancements in CL eld and the e ciency of the Binary Neural Networks (BNN), that use 1-bit for weights and activations to e ciently execute deep learning models. We propose a hybrid quantization of CWR* (an e ective CL approach) that considers di erently forward and backward pass in order to retain more precision during gradient update step and at the same time minimizing the latency overhead. The choice of a binary network as backbone is essential to meet the constraints of low power devices and, to the best of authors' knowledge, this is the rst attempt to prove on-device learning with BNN. The experimental validation carried out con rms the validity and the suitability of the proposed method.
2024
Image Analysis and Processing - ICIAP 2023 Workshops. ICIAP 2023.
39
50
On-Device Learning with Binary Neural Networks / Lorenzo Vorabbi, Davide Maltoni, Stefano Santi. - STAMPA. - 14365:(2024), pp. 39-50. (Intervento presentato al convegno Image Analysis and Processing - ICIAP 2023 Workshops. ICIAP 2023 tenutosi a Udine nel 09/2023) [10.1007/978-3-031-51023-6_4].
Lorenzo Vorabbi, Davide Maltoni, Stefano Santi
File in questo prodotto:
File Dimensione Formato  
On_device_learning_with_binary_neural_networks+-+camera+ready.pdf

embargo fino al 24/01/2025

Tipo: Postprint
Licenza: Licenza per accesso libero gratuito
Dimensione 1.3 MB
Formato Adobe PDF
1.3 MB Adobe PDF   Visualizza/Apri   Contatta l'autore

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/954673
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact