Streaming high-speed cameras pose a major challenge to distributed cyber-physical and IoT systems, because large data volumes need to be transferred under stringent realtime constraints. Edge processing can mitigate the data deluge by extracting relevant information from image data on-device with low latency. This work presents an FPGA-based 20 kfps streaming camera system, which can classify regions of interest (ROI) within a frame with a binarized neural network (BNN) in realtime streaming mode, achieving massive data reduction. BNNs have the potential to enable energy-efficient image classifications for on-device processing. We demonstrate our system in a case study with a simple real-time BNN classifier achieving 19.28 us latency at 0.52 W power consumption and resulting in a 980x data reduction. We compare external image processing with this result, showing 3x energy savings, and discuss the used HDL/HLS design flow for BNN implementation.

Jokic, P., Emery, S., Benini, L. (2018). BinaryEye: A 20 kfps Streaming Camera System on FPGA with Real-Time On-Device Image Recognition Using Binary Neural Networks. Institute of Electrical and Electronics Engineers Inc. [10.1109/SIES.2018.8442108].

BinaryEye: A 20 kfps Streaming Camera System on FPGA with Real-Time On-Device Image Recognition Using Binary Neural Networks

Benini, Luca
2018

Abstract

Streaming high-speed cameras pose a major challenge to distributed cyber-physical and IoT systems, because large data volumes need to be transferred under stringent realtime constraints. Edge processing can mitigate the data deluge by extracting relevant information from image data on-device with low latency. This work presents an FPGA-based 20 kfps streaming camera system, which can classify regions of interest (ROI) within a frame with a binarized neural network (BNN) in realtime streaming mode, achieving massive data reduction. BNNs have the potential to enable energy-efficient image classifications for on-device processing. We demonstrate our system in a case study with a simple real-time BNN classifier achieving 19.28 us latency at 0.52 W power consumption and resulting in a 980x data reduction. We compare external image processing with this result, showing 3x energy savings, and discuss the used HDL/HLS design flow for BNN implementation.
2018
2018 IEEE 13th International Symposium on Industrial Embedded Systems, SIES 2018 - Proceedings
1
7
Jokic, P., Emery, S., Benini, L. (2018). BinaryEye: A 20 kfps Streaming Camera System on FPGA with Real-Time On-Device Image Recognition Using Binary Neural Networks. Institute of Electrical and Electronics Engineers Inc. [10.1109/SIES.2018.8442108].
Jokic, Petar; Emery, Stephane; Benini, Luca
File in questo prodotto:
Eventuali allegati, non sono esposti

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/677179
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 23
  • ???jsp.display-item.citation.isi??? 22
social impact