Sovereign AI Platform on STACKIT: The Architecture of CompanyGPT Sovereign – Kubernetes, Keycloak, AI Model Serving & AI Governance

Since the German federal government awarded the contract for a sovereign AI platform for federal administration in May 2026 – with STACKIT as the cloud foundation – the term sovereign AI platform appears in almost every public tender. What is often missing is the architectural layer: which services run where, how they interlock, where identities and permissions live, how a model receives its requests, and how all of this can be demonstrated to data protection, information security and AI Act auditors.

This article answers exactly that for CompanyGPT Sovereign, the STACKIT variant of our AI platform CompanyGPT, which is now listed on the STACKIT Marketplace. It is written for IT leaders, architects, data protection and information security officers in public administration, at operators of government applications, and in mid-sized companies.

A shorter overview of the motivation is already available on innfactory.de. This article is about the target architecture – and how AI governance is anchored in it.

What a sovereign AI platform is – and what it is not

A sovereign AI platform is an AI application environment in which the operator, the data center, data storage and model operation are subject to a single legal jurisdiction and the customer retains control over identities, data, models, permissions and configuration.

Three distinctions help with classification:

Sovereignty is not a region setting. A European region of an international provider solves the question of data residency. It does not automatically answer the question of the law applicable to the operator, of control over the identity provider, or of where inference takes place. For many organizations an EU region is exactly the right answer – in that case CompanyGPT on Azure is the fitting variant. Sovereignty in the strict sense only emerges when hosting, identity, data and models are considered together.

Sovereignty is not the same as on-premises. Your own data center offers maximum control, but also maximum operational responsibility: GPU procurement, patching, high availability, 24/7 operations. A sovereign cloud like STACKIT shifts this responsibility to a managed-service provider in the same jurisdiction. For most administrations and mid-sized companies, that is the practical path.

Sovereignty is an architectural property. It cannot be bolted on afterwards. That is why we describe it here along the components the platform is made of. For the fundamentals: Digital Sovereignty and AI in Europe and the European Sovereign Stack Standard (ES3).

What public administration and government application operators require

The requirements for an AI platform in the public sector are more concrete than “GDPR-compliant”. Five points recur across our projects and tenders:

1. Verifiable data residency and operator jurisdiction. STACKIT operates its data centers in Germany (region EU01) and holds a BSI C5 Type 2 attestation as well as an ISO 27001 certification based on the BSI IT-Grundschutz methodology. This does not replace your own protection-needs assessment, but it provides the basis on which a government application may be connected at all.
2. Identities and permissions under your own control. Many authorities run their own Active Directory or a state-level identity provider and do not want an additional cloud tenant as a prerequisite for an AI assistant. That is why Keycloak is our default and Entra ID the option.
3. Models without third-country transfer. For professionals bound by secrecy obligations under Section 203 of the German Criminal Code, for social data or personnel files, inference must happen in the same jurisdiction as data storage. STACKIT AI Model Serving delivers that; international models remain an optional path released per role.
4. Auditability and cost transparency. Who used which model with which data, when, and what did it cost? Audit logs, cost centers and role-based gating are a prerequisite for administrations, not a convenience feature.
5. Reproducible infrastructure. An auditor wants to read the target state. Infrastructure as code and automated delivery via GitOps are therefore mandatory.

The context is broader than a single platform: with the Deutschland-Stack, state-level assistants such as F13 in Baden-Württemberg and the GovTech Germany framework agreement in which STACKIT participates, an ecosystem is emerging that a platform has to fit into.

The architecture at a glance

CompanyGPT Sovereign consists of the same application modules as the Azure variant – chat interface, companyRAG document search, companyFILES, administration and cost dashboard, sovereign web search, optionally AI Gateway and n8n automation – but on infrastructure operated entirely within STACKIT:

The following sections walk through the layers from runtime to governance.

Runtime: containers on STACKIT Kubernetes Engine

All application components run as containers on the STACKIT Kubernetes Engine (SKE). Kubernetes is not an end in itself here, but the layer that satisfies three requirements at once: isolation between components, declarative description of the target state, and automated maintenance by STACKIT.

Each application module lives in its own namespace with its own network policy. Incoming traffic is denied by default and only allowed from within the cluster; from the outside there is exactly one path – via the STACKIT Load Balancer, TLS termination and the authenticated gateway route. Certificates are issued and renewed automatically. SKE updates Kubernetes versions and operating system images in a defined maintenance window, node by node, without taking applications offline.

The platform is single-tenant: every customer receives their own cluster with their own databases. There is no shared control plane between tenants – for administrations this is frequently the prerequisite for connecting a government application at all.

This dedicated stack is also the affordable entry point: a pilot starts with a compact cluster, single-node databases and usage-based billing without per-seat fees. The same architecture is prepared for high-scale workloads – node autoscaling in SKE, horizontal scaling of the application modules, switching the databases to high-availability replication, practically unlimited Object Storage and model serving that STACKIT scales for you. From 50 users in one department to several thousand users across a state administration, the configuration changes – the platform does not.

Data layer: managed services in Germany

The data layer deliberately rests on STACKIT managed services so that backup, patching, encryption and high availability sit with the cloud provider rather than with you:

• STACKIT PostgreSQL Flex holds identities, sessions, permissions and the metadata of the document search – separated per application module with dedicated databases and credentials.
• STACKIT MongoDB Flex stores chat histories, agent configurations and user profiles with snapshots and point-in-time recovery.
• STACKIT Object Storage (S3-compatible) takes the documents of the knowledge base and backups of the cluster volumes; each purpose gets its own bucket with its own credentials.
• Persistent volumes inside the cluster – for the search index or shared files, for example – are replicated and regularly backed up to Object Storage.

The databases are released via IP access lists exclusively for the cluster’s address ranges and optionally defined admin networks; the lists are versioned together with the infrastructure, and every connection is TLS-encrypted and authenticated. Object Storage and Block Storage encrypt server-side with AES-256, MongoDB Flex comes with encryption at rest, and for PostgreSQL Flex customer-managed keys can be used via the STACKIT Key Management Service. In a purely sovereign tenant, data never leaves region EU01.

Connecting government applications: STACKIT VPN

An AI assistant only becomes useful to an administration once it knows the organization’s own sources: the document management system, file shares, SharePoint or Nextcloud, the staff directory – and, in the longer run, the government applications themselves. These systems usually sit in the organization’s own data center or with a municipal IT provider.

For this, the platform is connected via STACKIT VPN, a managed IPsec gateway for encrypted site-to-site tunnels between your own infrastructure and the STACKIT cloud. This lets companyRAG connectors reach internal file sources, allows Keycloak to use an existing Active Directory or LDAP as the user source, and lets n8n workflows talk to line-of-business applications – without opening a single port to the internet. How such a tunnel is set up and operated redundantly is described on our sister company’s product page for STACKIT VPN.

The result is a platform that behaves like an internal system for its users: sign-in with the familiar account, access only to the sources released for their role, and data that never leaves the controlled jurisdiction.

Network security today – and the next step: STACKIT VPC and Private Endpoints

The security architecture follows the principle of defense in depth. From the outside there is exactly one entry point: the STACKIT Load Balancer with TLS termination and authentication at the gateway. Inside the cluster, a default-deny policy blocks all incoming traffic that is not explicitly allowed. Cluster and administrators reach the managed databases through authenticated, TLS-encrypted endpoints that are restricted via IP access lists to the cluster’s address ranges. Credentials and model tokens live as Kubernetes Secrets in their respective namespace and are rotated automatically. Your own systems are connected via STACKIT VPN without opening a single port to the internet.

STACKIT has announced the next step for 2027: a VPC feature with Private Endpoints, through which managed services such as PostgreSQL Flex, MongoDB Flex and Object Storage become reachable exclusively via private network addresses without a public IP. Since all data access already originates from the cluster network today, the migration is a configuration change for the platform, not a rebuild – and the access lists are replaced by true network isolation. Organizations with higher requirements today can additionally use STACKIT Confidential Kubernetes for confidential workloads and customer-managed keys via the STACKIT Key Management Service.

Identity and permissions: Keycloak as default, Entra ID as option

In public-sector projects, the identity question often decides whether a project starts at all. Our answer: all application modules – chat, companyRAG, companyFILES, administration and AI Gateway – talk to a neutral OpenID Connect interface. Which identity provider sits behind it is configuration, not architecture.

Keycloak is the default. It runs inside the platform on SKE, uses PostgreSQL Flex as its database and provides OpenID Connect with Authorization Code Flow and PKCE. Password policy, brute-force protection and session lifetimes are pre-configured; multi-factor authentication via OTP and passkeys (WebAuthn) are enabled per tenant in the realm. For authorities with an existing directory service, Keycloak is also the connection point: user federation against Active Directory or LDAP as well as identity brokering to Entra ID, AD FS or academic federations such as DFN-AAI are standard features and are set up per tenant.

Entra ID is the option. If an organization already works in Microsoft 365, Entra ID is enabled through configuration – either with an app registration managed by us or, as is common in public administration, with a centrally approved registration owned by the customer. One feature strictly requires Entra ID: the Microsoft 365 add-in for Outlook and Teams. Everything else runs fully with Keycloak.

Groups become permissions. Regardless of the provider, group memberships are written into the token and resolved into roles within the platform. Roles control who is an administrator, which models and agents a user can see, which knowledge sources are available to them and what budget they are entitled to. The companyRAG document search checks every access against the identity provider, and knowledge sources are assigned to roles: a caseworker searches through the AI only the sources released for their role.

Model serving: sovereign by default, EU models per role

The model layer is where sovereignty shows most clearly. A CompanyGPT tenant can run exclusively on STACKIT AI Model Serving: chat, agents, web search, memory, file upload and document search all work with open-weight models from German data centers. Currently available there are, among others, Llama 3.3 70B, Gemma 4 31B, GPT-OSS 120B and 20B, Qwen3-VL 235B and embedding models such as E5-Mistral-7B (as of September 2026; the list changes regularly). Access is via an OpenAI-compatible API, so no application component needs vendor-specific code.

The access tokens for model serving are managed by the platform and rotated automatically; no user and no administrator handles model keys.

Inference inside the cluster – on self-operated GPU nodes, for example – is deliberately not part of the standard architecture. Managed model serving in a C5-attested environment gives most customers a better balance of operational effort, availability and verifiability. Anyone who needs a specific model for domain reasons can connect it.

For tenants that additionally require international models, there is a documented second path we sketched in an earlier article: OpenAI models via Azure AI Foundry in EU regions, and Gemini and Claude via Google Vertex AI in EU regions. These models are activated per tenant and released per role. A purely sovereign tenant simply does not contain them.

AI governance: budgets, traceability, guardrails

Sovereignty without governance is just hosting. That is why the control layer is an integral part of the architecture – through the administration module and, where more is needed, the AI Gateway:

• Model access by role. Which models a group may use is a policy, not a user decision. A team handling social data gets sovereign models only; a communications team may additionally use an EU model for translations.
• Budgets and cost centers. Consumption is recorded per user, team and cost center in euros; budgets cascade from the organization down to the individual agent. Overruns are reported or blocked.
• Audit logs. Every request is logged with user, model, timestamp and cost. This is the basis for the AI Act’s logging obligations as well as for internal audit.
• Guardrails. Model allowlists per role, budgets and guardrails are set centrally via the AI Gateway; agents and knowledge sources are released by administrators and assigned to roles.
• Transparency for users. The interface shows which model and which sources an answer was based on – a prerequisite for staff to interpret AI output correctly.

For data protection and information security this means, concretely: data processing within German jurisdiction, data residency in EU01, encryption, deletion concepts per tenant, a permission model derived from your own directory service, logging – and with that, the building blocks a data protection impact assessment and an IT-Grundschutz protection-needs assessment require.

What the EU AI Act requires from public bodies – and how the platform helps

A general AI assistant is usually not a high-risk system under the AI Act. Nevertheless, deployers – and public bodies in particular – face concrete obligations that a platform has to support:

• AI literacy (Art. 4). Staff must be competent in handling AI. Our AI training and the AI Officer qualification cover this; the platform supports it with usage policies and transparency notices directly in the interface.
• Transparency (Art. 50). Users must be able to recognize that they are interacting with an AI system and which content is AI-generated. This is built into the interface.
• Deployer obligations for high-risk use (Art. 26, applicable since 2 August 2026). If the assistant is used in an Annex III context, such as decisions about public benefits, human oversight, log retention and use according to the provider’s instructions apply. Audit logs and role gating provide the technical basis.
• Fundamental rights impact assessment (Art. 27). Public bodies must carry it out before high-risk deployment. The architecture description, the permission model and per-request model attribution are the documents needed for it.

The classification itself remains the authority’s responsibility. Our AI compliance consulting describes how to approach this in a structured way; the article NIS2, critical infrastructure and the EU AI Act covers the connection with NIS2 and KRITIS.

Operations: infrastructure as code and GitOps

The entire platform – cluster, databases, network, identity provider, application modules and model access – is described as code and rolled out automatically via GitOps. There is no click deployment and no manual configuration. Every change is a reviewed commit; every state can be reconstructed; a new tenant is created from the same modules with its own configuration file.

For public-sector customers this is more than DevOps convenience. An auditor working to ISO 27001, IT-Grundschutz or C5 reads the target state instead of asking for it, and changes to security-relevant settings are fully documented. Feature modules such as companyRAG, AI Gateway or n8n are switched on or off per tenant – an administration that only needs chat and document search gets exactly those components. Scaling is configuration of the same code as well: node count and autoscaling limits, database replication and replicas of the application modules grow with demand without the platform being set up anew.

Backups run on two levels: the managed databases back up daily with snapshots and point-in-time recovery, and the cluster volumes are backed up regularly to STACKIT Object Storage. We take care of platform maintenance and updates; model and security updates are rolled out through the same GitOps path as everything else.

Sovereign or Azure: freedom of choice, not either-or

CompanyGPT comes in two equivalent editions: on Microsoft Azure and as the Sovereign variant on STACKIT. Both use the same application layer, the same governance and the same permission model. The decision is not a question of better or worse, but of requirements – data residency, identity landscape, Microsoft 365 integration and model needs:

For organizations that work in Microsoft 365 and for which an EU region meets their requirements, Azure is the natural choice – with deep integration into Entra ID, Outlook and Teams. Those that need German data residency, C5 evidence, their own identities or models without third-country involvement choose Sovereign. Both paths are fully fledged, and a later switch remains possible. Our guide to enterprise AI platform selection criteria 2026 describes how to approach such a selection in a structured way.

Conclusion

A sovereign AI platform is the result of many individual decisions: containers on STACKIT Kubernetes Engine instead of self-managed clusters, managed databases in Germany, a single authenticated entry point, STACKIT VPN for connecting government applications and directory services, Keycloak as the default with Entra ID as an equivalent option, STACKIT AI Model Serving as the only mandatory backend, permissions derived from your own directory, budgets and audit logs as the governance layer – and all of it as code, rolled out automatically via GitOps.

CompanyGPT Sovereign is listed on the STACKIT Marketplace and billed on a usage basis. If you want to check whether your government applications, directory service and protection needs fit this architecture, get in touch – an architecture workshop is usually enough to settle the variant, identity path and model strategy.