GAP8

Starting in the Spring of 2020, I worked on building super low-energy computer vision systems. We used the GAP8 RISC-V microcontroller, which can accelerate 3x3 convolutional kernels using less than a watt of power. We took a split-execution approach where images from its grayscale camera were processed onboard by 6 convolutional layers before having the intermediate features sent to a laptop, which completed the rest of the model. That work became CLIO, where we showed that such a split-execution approach enables a better trade-off between latency and energy usage.

Into Winter 2022, I moved from classification to object detection and tracking. We further explored communication-aware training, which enabled even more efficient communication by transmitting only a learned subset of the intermediate features. In this version we transmitted the features wirelessly to a Jetson Nano edge GPU for an even more energy-efficient system overall.