Pushed by the fast exponential growth of machine learning models, low-precision floating-point (FP) formats, such as FP8 and FP16, are now supported by many commercial hardware platforms. Thanks to the available hardware support and their reduced storage and energy footprint, these low-precision formats are currently being investigated for many applications beyond neural network (NN) training and inference. These data types, however, rely on narrow exponent bitwidths, which directly translate to small dynamic ranges. Consequently, they are less robust to overflow with respect to FP32, especially during long accumulations. While overflowing values are often saturated in NN algorithms, this approach might not be sustainable in all scenarios, such as in the case of safety-critical applications. In this work, we propose a low-overhead hardware-software approach for overflow recovery. We devise an online recovery scheme, which leverages a RISC-V instruction set architecture (ISA) extension to minimize the overhead required to detect overflow and adjust the accumulation precision. For this purpose, branch instructions depending on the FP overflow flag and widening dot-product instructions working on 8-bit inputs and accumulating with 32 bits are added to a RISC-V core with mixed-precision capabilities. Our ISA extension adds less than 1% of hardware overhead to the RISC-V core and allows for less than 2% of performance penalty for overflow detection in a 128 x 128 matrix multiplication. Supporting overflow detection and recovery introduces negligible overhead with respect to a fragile baseline mixed-precision computation while maintaining its storage and performance advantages with respect to the full-precision baseline.

Bertaccini, L., Shen, S., Hoefler, T., Benini, L. (2024). Extending RISC-V for Efficient Overflow Recovery in Mixed-Precision Computations. Institute of Electrical and Electronics Engineers Inc. [10.1109/iccd63220.2024.00048].

Extending RISC-V for Efficient Overflow Recovery in Mixed-Precision Computations

Bertaccini, Luca;Benini, Luca
2024

Abstract

Pushed by the fast exponential growth of machine learning models, low-precision floating-point (FP) formats, such as FP8 and FP16, are now supported by many commercial hardware platforms. Thanks to the available hardware support and their reduced storage and energy footprint, these low-precision formats are currently being investigated for many applications beyond neural network (NN) training and inference. These data types, however, rely on narrow exponent bitwidths, which directly translate to small dynamic ranges. Consequently, they are less robust to overflow with respect to FP32, especially during long accumulations. While overflowing values are often saturated in NN algorithms, this approach might not be sustainable in all scenarios, such as in the case of safety-critical applications. In this work, we propose a low-overhead hardware-software approach for overflow recovery. We devise an online recovery scheme, which leverages a RISC-V instruction set architecture (ISA) extension to minimize the overhead required to detect overflow and adjust the accumulation precision. For this purpose, branch instructions depending on the FP overflow flag and widening dot-product instructions working on 8-bit inputs and accumulating with 32 bits are added to a RISC-V core with mixed-precision capabilities. Our ISA extension adds less than 1% of hardware overhead to the RISC-V core and allows for less than 2% of performance penalty for overflow detection in a 128 x 128 matrix multiplication. Supporting overflow detection and recovery introduces negligible overhead with respect to a fragile baseline mixed-precision computation while maintaining its storage and performance advantages with respect to the full-precision baseline.
2024
Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors
268
275
Bertaccini, L., Shen, S., Hoefler, T., Benini, L. (2024). Extending RISC-V for Efficient Overflow Recovery in Mixed-Precision Computations. Institute of Electrical and Electronics Engineers Inc. [10.1109/iccd63220.2024.00048].
Bertaccini, Luca; Shen, Siyuan; Hoefler, Torsten; Benini, Luca
File in questo prodotto:
Eventuali allegati, non sono esposti

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/1004838
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact