The University of Münster launched uniGPT in May 2024 as an on-premises chatbot for its members. A paper in the EUNIS 2026 proceedings reviews nearly two years of operation (May 2024 to February 2026) and eight design iterations. The motivation was data privacy and GDPR compliance, the cost of per-seat LLM subscriptions, and vendor lock-in.
Foundation. uniGPT runs on an existing Kubernetes cluster built on OpenStack, reusing GPUs from earlier institutional projects. Hardware grew from a single NVIDIA A40 node with four GPUs to a mix of A40, A100 and L40S nodes. All deployments follow a GitOps workflow with Helm, Kustomize and ArgoCD, so a new model or service is added by changing Helm charts, in several cases with unmodified upstream Docker images.
API gateway. The first version served open-weight models (initially Llama 2 and Mixtral) through Hugging Face Text Generation Inference (TGI) and the ChatUI frontend. When demand shifted to programmatic access, the team added LiteLLM as a proxy layer, because TGI had no native OpenAI-compatible interface. LiteLLM provides the OpenAI-compatible API, routing, API keys and quotas. The API became visible to researchers in September 2024.
Inference engine. The team replaced TGI with vLLM. Under peak load TGI suffered VRAM exhaustion and KV-cache fragmentation that caused persistent out-of-memory errors, and it lacked robust native function calling. The migration took less than a week and was phased: new models were deployed on vLLM while legacy models were retired through the normal lifecycle.
Other modalities. Image generation with Flux runs on the Hugging Face Diffusers library, and speech recognition uses NVIDIA Canary, selected after a multi-language benchmark against Parakeet and Whisper. Both share the LLMs' GPU pool. For retrieval-augmented generation the central team offers the LLM and an embedding model (Qwen3-Embedding-8B) as a service, while departments host their own middleware, for example built with LangChain, and vector databases on the cluster. Projects on this offering include TutorAI for lectures and Quin, an IT support chatbot.
Frontend and external models. ChatUI was replaced by LibreChat to support reasoning models and tool calling; it was preferred over OpenWebUI for licensing reasons. Through GÉANT OCRE contracts, Gemini and Anthropic models on Google Cloud and OpenAI models on Microsoft Azure were added alongside the on-premises models, and sensitive data can be routed only to on-premises models. Via the API, uniGPT is also embedded in the learning management system, the content management system and a translation tool that uses the official university glossary.
Usage. API calls now account for over 99% of interactions, with peaks of 1.66 million local inference requests per day, from researchers in the humanities, law, biology and psychology as well as computer science. One full-time position is dedicated to the AI services.