MiniFloat-NN and ExSdotp: An ISA Extension and a Modular Open Hardware Unit for Low-Precision Training on RISC-V Cores

Low-precision formats have recently driven major breakthroughs in neural network (NN) training and inferenceby redncing the memory footprint of the NN models and improving the energy elIiciency of the underlying hardware arehitectures, Narrow integer data types have been vastly investigated for NN inference and have successfully beeo pnsbed to the extreme of ternary and binary representations. In contrast, most training-oriented platforms use at least 16-bit floating-point (FP) Cormats. Lower-precision data types such as 8-bit FP formats and mixed-precision techniques have only recently been explored in hardware implementations. We present MiniFloat-NN, a RISe-v instruction set architecture extension for low-precision NN training, providing support Cor two 8-bit and two 16-bit FP Cormats andexpanding operations. The extension includes sum-of-dot-product instructions that accnmulate the result in a larger format and three-term additions in two variations: expanding and non-expanding. We implement an ExSdolp unit to elliciently support in hardware both instruetion types. The fused nature of the ExSdotp module prevents precision losses generated by the non-assofiativity of two consecutive FP additions while saving around 30% of the area and eritieal poth compared to a cascade of two expanding fused multiply-add units. We repUcate the ExSdolp module in a SIMD wrapper and integrate It into an open-source floating-point unit, which, conpled to an open-source RISC-V core, lays the foundation for future scalable architectures targeting low-precision and mixed-precision NN training. A cluster containing eight extended oores sharing a scratehpad memory, implemented in 12nm FinFET technology, achieves up to 575 GFLOPS/W when computing FP8-to-FP16 GEMMs at 0.8 V, 1.26GHz.