PACE: An Optimal Piecewise Polynomial Approximation Unit for Flexible and Efficient Transformer Non-linearity Acceleration

Diverse Transformer models are being explored for Contextual and Generative AI. While matrix-matrix and matrixvector multiplication remain invariant core operators, non-linear activation functions change and evolve very rapidly, emerging as a new bottleneck - especially as matrix-matrix and matrixvector multiplication are increasingly accelerated. This necessitates hardware flexibility to accelerate diverse non-linear operators, as software emulation is not sufficiently fast and efficient. We present PACE an open-source Polynomial Approximation Compute Engine that accelerates non-linear functions using optimal Piecewise Polynomial Approximation (PwPA) with arbitrary degrees and partitions, achieving <0.2 % accuracy loss on state-ofthe-art (SoA) pretrained Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and Large Language Models (LLMs) without fine-tuning. Integrated as an ISA extension into a RISCV processing cluster, PACE incurs 14% area overhead of the cluster while delivering up to 1 FP32 PolyEval per cycle per core at 0.95 GHz with an efficiency of 8 pJ/PolyEval. The PACEaugmented cluster achieves a 750× speedup over the C math.h library and 65× over software-based PwPA. PACE also delivers the highest throughput (1.5×) compared to existing generalpurpose approximation hardware.