Hyperdrive: A systolically scalable binary-weight CNN Inference Engine for mW IoT End-Nodes

Deep neural networks have achieved impressive results in computer vision and machine learning. Unfortunately, state-of-The-Art networks are extremely compute-And memory-intensive which makes them unsuitable for mW-devices such as IoT end-nodes. Aggressive quantization of these networks dramatically reduces the computation and memory footprint. Binary-weight neural networks (BWNs) follow this trend, pushing weight quantization to the limit. Hardware accelerators for BWNs presented up to now have focused on core efficiency, disregarding I/O bandwidth and system-level efficiency that are crucial for deployment of accelerators in ultra-low power devices. We present Hyperdrive: A BWN accelerator dramatically reducing the I/O bandwidth exploiting a novel binary-weight streaming approach, and capable of handling high-resolution images by virtue of its systolic-scalable architecture. We achieve a 5.9 TOp/s/W system-level efficiency (i.e. including I/Os)-2.2x higher than state-of-The-Art BNN accelerators, even if our core uses resource-intensive FP16 arithmetic for increased robustness.