Neural Network Dependability Kit (NNDK)
Diabetic retinopathy (DR), a complication of the retina brought about by diabetes, is one of the most frequent reasons for vision loss in European adults between 25 and 60 years of age. When detected early, treatment can effectively reduce or prevent vision loss. To date however, national screening programs have been available in only a few countries and even then, they are costly.
As part of the FED4SAE innovation measure supported by fortiss, Ubotica Technologies emerged as the winner of an open call and has developed a deep learning-based solution for detecting the presence of DR indicators in retinal images. These DR indicators can be made recognizable with the help of special retina and fundus cameras.
By employing the Neural Network Dependability Kit (NNDK) software developed by fortiss, researchers were able to improve the deep learning model so that it can be directly utilized in a fundus camera. The solution was developed to help ophthalmologists consistently and precisely assess and diagnose DR. For this purpose, and with the help of NNDK, the solution also supplies information regarding the foundation on which the assessment was made.
The aim of the development was to create a working prototype that demonstrates the classification of retinal fundus images for the presence of indicators of DR. However, the inherent uncertainties of machine learning algorithms, due to their data-driven approach, limit their integration into such a classification system, particularly in such a safety-critical field.
Key challenges in the deployment of artificial intelligence (AI)-based solutions include, on the one hand, being able to make statements about the reliability of the decisions made by artificial neural networks (ANNs) in terms of robustness, interpretability and correctness, and, on the other hand, ensuring the continuous monitoring of the neural network’s decisions.
Existing AI models faced the particular challenge of providing the transparency and traceability required for medical diagnoses. For clinical integration, robust methods therefore had to be developed to better address trust, safety and regulatory requirements.
Ubotica has trained a deep learning-based neural network to detect DR, using 35,000 freely available, pre-classified retinal images. The technical solution utilises the Intel Movidius Myriad X Vision Processing Unit (VPU), an electronic processing unit that can be integrated directly into the fundus camera. First, the VPU pre-processes the retinal images using its integrated image filtering functions. The trained neural network is then applied to examine the retinal image for DR indicators. NNDK is used in the development of the neural network to evaluate its performance using various metrics and thus further improve the model.
The NNDK is an open-source toolbox developed by fortiss to support safety engineering and to improve understanding of neural networks. It supports verification, test case generation and the calculation of metrics for neural networks. It is based on formal methods used for modelling and rigorously verifying computer systems, in order to prove faultlessness in safety-critical systems and to demonstrate the quality of neural networks. This enables the development of neural networks that are more robust, reliable, interpretable and trustworthy.
In the development of DR detection, particular use was made of NNDK’s reliability metrics and its support for runtime monitoring: for example, by employing a specific metric to calculate neuron coverage, the size of the neural network was reduced by 74% without significantly affecting its accuracy in classifying retinal images. This significantly accelerated the calculation of classification results. Furthermore, NNDK’s runtime monitoring function was used to display to the camera user those training images that most closely correspond to the analysed image. This allows the examiner to independently verify the AI system’s decision once again, thereby extending the in-line detection of diabetic retinopathy with a verifiable AI element.
![[Translate to English:] Icon Loesung](/fileadmin/user_upload/Icons/Icons-Neu/fortiss-icon-loesung.png){kind=icon}
The solution that was developed is not aimed at replacing medical experts with the screening of the retinal fundus images, rather to support the role of the examining physician through the creation of an initial classification. What’s also crucial is that fortiss added an explanation component for medical professionals, which reinforces trust in the decision made by the AI technology. The software thus helps ophthalmologists precisely and consistently assess and diagnose DR.
Ubotica is working together with retina camera manufacturers and the Irish public health service to establish the solution on the market. The function in the NNDK software that monitors the decisions made by the ANNs offers a significant advantage for solution integrators. The utilization of Ubotica’s technology will considerably improve the efficiency of the cameras, as well as the ophthalmologist’s approach.
The project demonstrates that the early integration of safety engineering principles into the development of AI-based systems is crucial for their deployment in regulated and safety-critical sectors. In particular, the combination of technical performance, interpretability and transparent decision-making mechanisms strengthens healthcare professionals’ confidence in AI-supported diagnostic systems and facilitates their practical implementation in clinical settings.
Ubotica Technologies is small-to-medium Irish enterprise that specializes in AI solutions for edge-based computer vision applications. The company maintains development centers at several European locations that collaborate with research centers to spur innovations for the target markets. The primary target markets include aerospace, high-speed motion tracking and medical image analysis.
As part of the FED4SAE programme, Ubotica collaborated with research partners to develop innovative AI applications for safety-critical applications.
