Local, flexible, secure large language models for small and medium-sized businesses
SH-LLM
Project description
Running LLMs locally is technically demanding. It requires specialized hardware, deep system integration knowledge, and expertise in model architecture, which most SMEs lack. At the same time, using cloud-based LLMs raises serious privacy and confidentiality concerns—especially when handling sensitive data or operating under strict regulations like GDPR. Off-the-shelf APIs are also expensive and poorly suited for domain-specific use cases, leaving SMEs without practical, compliant, or cost-effective AI solutions.
The project addresses these challenges by locally hosting an open-source LLM (starting with LLama and DeepSeek) on fortiss hardware, allowing full control over data and infrastructure. This setup enables experimentation with different models, finetuning strategies, and deployment architectures tailored to SME needs.
Research contribution
- Ability to run SME-specific LLM workloads securely on fortiss hardware, ensuring data privacy, regulatory compliance, and full control over sensitive information
- A Tangible Demonstrator: A real-world prototype showcasing the capabilities of a self-hosted, business-oriented LLM system, used for engagement, acquisition, and internal learning
- Foundation for Industry Collaboration: A robust technical and organizational basis for future industry projects involving self-hosted AI, enabling tailored solutions and long-term partnerships with SMEs
Project duration
01.03.2025 - 30.06.2025
