A Heterogeneous System-on-Chip with an 83 GFLOp/s, 1.2 TFLOp/s/W Parallel Programmable Accelerator for Real-time On-device Learning in Autonomous Nano-UAVs

Miniaturized autonomous robotics, such as sub-10 cm nano-sized Unmanned Aerial Vehicles (UAVs), is a novel and pervasive technology gaining attention in both civil and industrial application domains. To enable full autonomy, tiny robots must execute multiple real-time workloads, ranging from precise Artificial Intelligence (AI)-based perception algorithms to high-throughput control pipelines, while leveraging their minimal onboard computational resources. At the same time, well-trained AI-based algorithms suffer from the domain shift problem, which leads to poor performance when deployed in real-world domains. This work presents AlSaqr, a heterogeneous and secure RISC-V System-on-Chip (SoC) that integrates a dual-core Linux-capable 64-bit host processor, an 8-core 32-bit Programmable Parallel Accelerator (PPA) embedding an FP8/FP16 tensor core, and a secure subsystem. Fabricated in GlobalFoundries 22nm technology, the proposed Pixhawk FMUv6-compliant SoC operates from 210MHz at 0.52V to 900MHz at 0.8V within a power envelope from 90mW to 635mW occupying 9mm2 silicon footprint, matching the tight size, weight, and power constraints of nano-UAVs. Post-silicon results show that the PPA achieves a peak FP16 throughput of 83GFLOp/s at 900MHz and a peak energy efficiency of 1.2 TFLOp/s/W at 210MHz. To demonstrate the SoC’s capabilities, we showcase a deep learning model for human pose estimation that produces low-level setpoints for the nano-UAV’s control loops. AlSaqr achieves a 40 frame/s inference, while concurrently fine-tuning the same model with on-device learning (i.e., backpropagation), enabling real-time on-the-fly adaptation without interruption of the primary mission.