Generating Event-Based Datasets for Robotic Applications Using MuJoCo-ESIM

Camilo Amaya, Gintautas Palinauskas, Evan Eames, Michael Neumeier and Axel von Arnim

Proceedings of the 2023 International Conference on Neuromorphic Systems, pp. 7

August 2023 · New York, NY, USA · Association for Computing Machinery · doi: 10.1145/3589737.3605984


Event-based cameras are cameras with high dynamic range that measure changes in light intensity at each pixel instead of capturing frames like traditional cameras. There are several event-based camera simulators for generating event-based datasets, each specialized towards a particular domain. However, none are designed specifically for robotic use-cases. This work addresses this issue using MuJoCo, a high performance physics engine; in combination with ESIM, an established event-generation method. To the authors' knowledge, this is the first robotic simulator tool for generating event-based datasets specifically designed for the robotics domain. Furthermore, to demonstrate its capabilities we generate an event-based visual dataset of industrial sockets, which is then used to train a SNN classifier.

subject terms: Neurorobotics, neuromorphic vision, ESIM, spiking classification, SLAYER, event-based dataset generation, industrial sockets, MuJoCO, spiking neural networks