With the SpikingBody project, fortiss is further expanding its leading expertise in biologically inspired AI technologies in the field of neuromorphic computing. Under the leadership of Michael Neumeier, the team of researchers explored the use of neuromorphic algorithms based on spiking neural networks (SNNs) and Intel's neuromorphic research chip Loihi for real-time action recognition. The research, funded by the Bavarian Research Foundation, opens up promising prospects for applications in human-machine interfaces, particularly in the fields of robotics, industrial automation, and interactive assistance systems.
Neuromorphic intelligence for intuitive human-machine interaction
Understanding human actions is fundamental to the safety, acceptance, and interactivity of robots and machines. However, effective recognition through human-machine interfaces requires high detection accuracy, fast response times, and low energy consumption. Conventional systems often reach their limits in this regard. Neuromorphic technologies, which are modeled on the architecture and functioning of the human brain, make it possible to optimize action recognition systems according to these criteria.
SpikingBody relies on a novel approach: combining event-based vision sensors with SNNs implemented on neuromorphic hardware. The event-based image sensors respond only to changes in the environment, rather than capturing complete images at high data rates. This reduces computational load and provides high temporal resolution with minimal latency. The captured sensor data is processed directly as spikes in an SNN, which runs energy-efficiently on Intel’s research platform Loihi. The network recognizes and interprets actions in real time – with the ability to learn new actions 'on the fly' without needing to retrain the entire system.
Neuromorphic AI is transforming motion analysis in tennis
This technology has been specifically applied to tennis action recognition and offers new possibilities for analyzing and responding to player movements in real time. Thanks to the neuromorphic approach, detection latency can be reduced to the point where the type of movement is recognized in real time. The event-driven camera, which generates a very compact image stream with virtually no latency, makes it possible to capture very fast movements without motion blur, thereby enabling reliable speed estimations.
The system developed by fortiss uses just a single event camera to recognize the type of a tennis player’s movement (such as serve, forehand, or backhand) in real time. Based on this recognition, initial conclusions can be drawn or more expensive, high-precision systems can be activated to enable more detailed analysis – for example, for broadcasting matches on television. At the same time, it offers sports clubs a more affordable and user-friendly alternative to professional analysis and training systems.
