Reducing latency in a converted spiking video segmentation network

Abstract

Spiking Neural Networks (SNNs) can be configured to produce almost-equivalent accurate Analog Neural Networks (ANNs) by various ANN-SNN conversion methods. Most of these methods are applied to classification and object detection networks tested on frame-based datasets. In this work, we demonstrate a converted SNN for image segmentation and applied to a natural video dataset. Instead of resetting the network state with each input frame, we capitalize on the temporal redundancy between adjacent frames in a natural scene, and propose an interval reset method where the network state is reset after a fixed number of frames. We studied the trade-off between accuracy and latency with the number of interval reset frames. We also applied layer-specific normalization and early stopping to speed up network convergence and to reduce the latency. Our results show that the SNN achieved a 35.7x increase in convergence speed with only 1.5% accuracy drop using an interval reset of 20 frames.