Fast and Energy-Efficient CNN Inference on IoT Devices Mohammad Motamedi ECE Department University of California, Davis Email: mmotamedi@ucdavis.edu Daniel Fong ECE Department University of California, Davis Email: dfong@ucdavis.edu Soheil Ghiasi ECE Department University of California, Davis Email: ghiasi@ucdavis.edu Abstract—Convolutional Neural Networks (CNNs) exhibit re- markable performance in various machine learning tasks. As sensor-equipped internet of things (IoT) devices permeate into every aspect of modern life, it is increasingly important to run CNN inference, a computationally intensive application, on resource constrained devices. We present a technique for fast and energy-efficient CNN inference on mobile SoC platforms, which are projected to be a major player in the IoT space. We propose techniques for efficient parallelization of CNN inference targeting mobile GPUs, and explore the underlying tradeoffs. Experiments with running Squeezenet on th
