SpikeClouds: Streaming Spike-Based Processing of LiDAR for Fast and Efficient Object Detection

abstract

LiDAR sensors are used to provide three-dimensional information about the environment in many robotics applications. The information, accumulated in 3D point clouds, is first acquired by the sensor and then processed further, which leads to high end-to-end latencies and large memory footprints. Streaming approaches tackle this problem by processing partial point cloud data during scanning of the environment. In contrast to existing work that is limited to power hungry, rotating mechanical scanners, in this letter, we present a streaming method for more efficient scanline-based LiDAR sensors. We process the sequence of scanlines in form of SpikeClouds with a Spiking Neural Network (SNN) backbone and perform 3D object detection from the accumulated information using a Convolutional Neural Network (CNN) head. Our method achieves close to state-of-the-art detection performance on datasets KITTI and JRDB22 while reducing the end-to-end latency by 10% and the average memory footprint by 95% on standard GPU hardware. Additionally, when ported onto neuromorphic hardware, our backbone requires 25× less energy compared to reference backbones. SpikeClouds achieves fast and efficient environmental perception for robotic applications by streaming LiDAR to enable spike-based processing.