Case study IGAISE – Impact of generative AI in software engineering
With the rise of Generative AI (GenAI) in day-to-day development, companies face the challenge of quantifying the actual added value of these tools. fortiss has developed a comprehensive framework for Siemens that goes beyond subjective perceptions and enables an objective assessment of the impact on the entire software development life cycle (SDLC).
Challenge
Modern software organisations today face the problem that, despite perceived benefits, the introduction of GenAI tools can sometimes have unexpected negative effects on overall software delivery performance. Whilst individual tasks such as code reviews are often perceived as more efficient, there is a lack of robust evidence in practice to support an increase in overall productivity. Measuring these effects is also highly complex, as there is a lack of pragmatic and context-specific evaluation methods to examine the impact of AI on critical dimensions such as software quality and maintainability in isolation. Without such a sound factual basis, companies risk not only inefficient investment in new tools, but also a creeping deterioration of their established development processes.
Solution
To address these challenges, fortiss and Siemens have developed a structured framework that combines academic rigour with direct industrial applicability. The solution provides detailed guidelines for the multidimensional analysis of GenAI impacts, with a particular focus on speed, automation, workflow, quality, maintainability and relevance. The methodological foundation is a comprehensive literature review, validated through continuous feedback from real-world research and transfer projects as well as industrial practice. This context-sensitive evaluation approach enables companies to make informed assessments of their AI tools and ensures that technological innovations are deployed specifically where they generate measurable business value.
Result
- Publication of a guide on measuring the efficiency and effectiveness of GenAI in software engineering
- Provision of concepts and recommendations to enable the objective assessment of AI impacts
- Holistic understanding of processes: In-depth insights into the impact of GenAI on all phases of the SDLC – from planning to maintenance
- Terminological clarity: Definition of key factors and terms for interpreting AI applications in various scenarios.
Outcome
The collaboration between fortiss and Siemens demonstrates that systematic performance measurement provides the necessary foundation for integrating Generative AI into industrial development processes in a way that adds value and is sustainable. fortiss supports companies in analysing their workflows and identifying opportunities for the targeted use of GenAI tools.
Project partner
