↓ Skip to main content
9 – 17 UHR +49 8031 3508270 LUITPOLDSTR. 9, 83022 ROSENHEIM
DE / EN

AI Gateway vs. TrueFoundry: Enterprise platform with control plane or infrastructure component in your own tenant?

Tobias Jonas Tobias Jonas | | 15 min read

Key facts (as of 26 September 2026):

  • TrueFoundry (Ensemble Labs Inc, San Francisco) is, per its own website, an AI gateway for enterprises with an LLM Gateway, MCP Gateway and Agent Gateway under a shared control plane, documented in seven Kubernetes deployment scenarios from pure SaaS to fully self-hosted; per the public repositories the gateway core is not published as open source, and the SSO documentation names an Enterprise plan.
  • The innFactory AI Gateway is deployed into the customer’s cloud tenant and bundles cost centers with per-member caps, budgets in euros and a companion that measures the agent time of Claude Code, Codex and OpenCode per repository, including subscription usage.
  • Per the deployment documentation, LLM traffic stays in your own infrastructure with a self-hosted gateway plane, while a global authentication and licensing server remains with TrueFoundry and, per the vendor, receives e-mail addresses and the request count; with the innFactory AI Gateway, login runs exclusively against the customer’s Entra ID or OIDC, and a local companion for agent time is not described in the TrueFoundry documentation (as of 26 September 2026).
  • TrueFoundry fits when you run Kubernetes on AWS, Azure or Google Cloud, want gateway, control plane and optionally model deployment from a single source and need SAML or SCIM; the innFactory AI Gateway fits when gateway, dashboard and data should live in your own tenant and cost centers with hard stops per member in euros and the agent time of Claude Code, Codex and OpenCode per repository are decisive.

TrueFoundry describes its product on its own website as “AI Gateway For Enterprises: Built-in Governance & Monitoring” and as “A unified interface for accessing 1000+ LLMs with enterprise-grade security, observability, and governance”. According to the website footer, the company behind the brand is Ensemble Labs Inc, based in San Francisco. By the vendor’s own account, the platform comprises an LLM Gateway, an MCP Gateway and an Agent Gateway under a shared control plane, plus a separate module for model deployment.

TrueFoundry is a solid product with an unusually well-documented range of deployment options: seven named scenarios from pure SaaS to a fully self-hosted platform, Terraform modules for AWS, Azure and Google Cloud, Helm charts, SAML, OIDC and SCIM, guardrails with several partners and its own MCP Gateway.

The real decision lies elsewhere: do you want to operate a platform whose control plane, login and license checks follow the vendor’s architecture - or an infrastructure component that runs as a container in your tenant, authenticates against your identity provider and additionally captures the agent time of your coding agents? This comparison lays out both, with sources, as of 26 September 2026.

What TrueFoundry is

TrueFoundry is the product brand of Ensemble Labs Inc, San Francisco. No change of ownership is documented on the vendor pages we reviewed.

According to the public repositories of the GitHub organisation, the gateway core is not published as open source. Per the deployment documentation, the runtime is pulled as a container image via a vendor-issued image pull secret and activated with a license key (TFY_API_KEY) and a tenant name. Publicly available on GitHub are the Terraform modules (Apache-2.0), the separate RAG framework Cognita (Apache-2.0) and a community-maintained model registry (MIT); the Helm chart repository infra-charts and the SDKs are public but carry no declared license. As a tier, the SSO documentation names a “higher-tier Enterprise plan”.

Operating model: SaaS, self-hosting or infrastructure in your own tenant

TrueFoundry documents seven deployment scenarios, all on Kubernetes - from “AI Gateway SaaS only” via “Gateway Plane only” and “Control Plane + Gateway Plane” to “Control Plane + Gateway Plane + Compute Plane”. According to the vendor, the gateway can be used without the deployment module and ships as a Helm chart.

The decisive point is who operates which component. For the self-hosted gateway plane, the vendor states: “All LLM traffic stays within your own infrastructure and TrueFoundry doesn’t come into the live path of a request to LLM.” The request path to the model therefore stays in your infrastructure. Even with full self-hosting, one component stays with the vendor: “In this case, everything except the authentication server and analytics server, everything is hosted on your own infrastructure.” The architecture documentation calls this building block the “Global Authentication/Licensing Server”, “used to authenticate every user who logs into the control plane”: “In case you are installing the control plane on your end, the Global Authentication server remains within TrueFoundry.” What is transmitted, according to the vendor, is “the emails of the employees logging into the control plane, and the number of requests flowing through the AI Gateway” - “This helps us keep track of the licenses and billing.” Per the SSO documentation, there is also the variant “Direct IdP (no TrueFoundry Auth Server)”, “On-prem only, available on the higher-tier Enterprise plan”.

The innFactory AI Gateway is cut differently: a Rust binary in a container plus a Next.js dashboard, deployed into your own cloud tenant - Azure Container Apps or AKS, STACKIT or any Kubernetes. Gateway, dashboard and data live in the tenant, and login runs against your Microsoft Entra ID or an OIDC provider such as Keycloak. The license is a monthly license by user count; user login runs exclusively against your identity provider. innFactory does not host the gateway as SaaS. It has been in production with first customers since August 2026; the current version 1.0.0-rc16 is a release candidate without a GA tag - we say that openly.

Identity, SSO and roles

Per the SSO documentation, TrueFoundry supports OpenID Connect, SAML 2.0 and SCIM against ten named IdPs, including Entra ID, Okta, Google Workspace and Keycloak. In the default model, the central auth server brokers the SSO flow: “Only the user’s email and a request count are sent to login.truefoundry.com - used for licensing and tenant routing.” The role model is resource-specific - for example Manager and User for model accounts, MCP Server Manager, User and Approver for MCP servers - and virtual accounts serve as service identities with token rotation.

The innFactory AI Gateway authenticates via Microsoft Entra ID or generic OIDC (Keycloak documented, AWS Cognito claims handled). Seven roles are included in every license tier: Owner, Admin, Auditor, Finance, Security, Member and Guest. IdP groups map to six roles, cost centers have their own administrators, and models, budgets and rate limits can be set per person. The innFactory AI Gateway does not currently offer SAML or SCIM - TrueFoundry is broader here.

Cost control and AI FinOps

TrueFoundry controls budgets through rules that, per the documentation, filter by “Subjects: Users, teams, or virtual accounts”, by models and by metadata from the X-TFY-METADATA header. Rules block or observe: “You can automatically block requests when limits are exceeded, or run in audit mode to monitor spending before enforcing hard limits.” Rate limits apply per minute, hour or day. Currency, the term cost center and the price source are not documented on the pages reviewed.

The innFactory AI Gateway works with independent budget gates: organisation (monthly), cost center (monthly plus daily and weekly windows, pool or per-user mode with per-member caps), user and API key. Each exhausted gate answers with HTTP 402 and error code E4006, optionally a budget fallback model kicks in; per-member caps are always hard stops. Added to this are TPM and RPM rate limits, alerts at 80 and 90 percent and a budget_exceeded event for webhooks. Costs are displayed in euros at the ECB daily rate, model prices come from ai-prices.eu with admin approval, and every buffered response carries the header x-aigateway-response-cost.

Coding agents, subscriptions and agent time

TrueFoundry addresses coding agents explicitly: the llms.txt names “Claude Code Governance” as a use case, the blog offers a governance guide for Claude Code, a Cline guide with budgets, rate limits and logs and a post on OpenCode; a change announcement covers Gemini CLI, and the repository tfy-claude-auth-token provides binaries for Claude authentication. What is measured is what passes through the gateway: requests, tokens and costs. We did not find dedicated pages for Codex CLI, Cursor or Copilot; MDM rollout is not documented. According to the TrueFoundry documentation (as of 26 September 2026), a local companion of this kind is not described.

The innFactory AI Gateway goes one step further here. In proxy mode, developers clone admin templates for six tools into their own setup with their own key and cost center: Claude Code, Codex, OpenCode, Cowork, VS Code Continue and VS Code Copilot (BYOK). In addition, the companion aigw - a background service for Linux, macOS and Windows - reads the local session logs of Claude Code, Codex and OpenCode and reports agent time, LLM time and token classes per repository, user, tool and cost center. It detects whether a session ran through the gateway or on the subscription (Claude Max, Codex plan); subscription usage is valued at list price but never billed or limited and is always shown separately. Rollout works via enrollment tokens through Intune, macOS policy or a Linux script; only a fixed allow-list is transmitted, without prompts. For context: the binaries are not code-signed yet, and there is no self-update yet.

Providers, dialects and models

TrueFoundry quotes different numbers depending on the page: “1000+ LLMs” in the gateway introduction and llms.txt, “1600+ models” on the product page, “15+ platforms” in the provider overview. Documented are, among others, Google Vertex AI, AWS Bedrock, Azure OpenAI, Azure AI Foundry, OpenAI, Databricks, OpenRouter and self-hosted models; the llms.txt additionally names Anthropic, Groq, Mistral and Cohere. STACKIT and IONOS are not documented. According to the vendor, the API is “One endpoint with an OpenAI-compatible schema for every provider”; native Anthropic or Gemini dialects and realtime WebSockets are something we could not confirm on the pages reviewed.

The innFactory AI Gateway supports 13 provider types, including OpenAI, Azure OpenAI, Azure AI Foundry, Anthropic (direct or via Vertex), Gemini (including the Vertex EU endpoint), Mistral, AWS Bedrock, STACKIT, IONOS, Ollama and ElevenLabs. Inbound it speaks three dialects - OpenAI, Anthropic and Gemini - plus realtime WebSockets for OpenAI, Azure OpenAI, ElevenLabs and Gemini Live. On the number of connected models, TrueFoundry is broader; on EU providers and native dialects, the innFactory AI Gateway sets different priorities.

Governance: guardrails, MCP, routing and Flow Builder

At TrueFoundry, guardrails work in two modes per the documentation - “Validate” inspects and blocks, “Mutate” rewrites - with three enforcement strategies: “Audit”, “Enforce” and “Enforce But Ignore On Error”. As integrations, the documentation lists Bedrock Guardrails, Google Model Armor, OPA and Cedar; as partners, the blog names Enkrypt AI, Prisma AIRS and Akto. The MCP Gateway offers, by the vendor’s own account, a central registry, “one token per user, auto-refreshed across all MCP servers”, role-based access down to tool level, Virtual MCP Servers, pre-built servers for Slack, Confluence, Sentry and Datadog and OpenAPI-to-MCP conversion. For routing, TrueFoundry documents three strategies - weight-, latency- and priority-based - bundled in Virtual Models with retry and fallback configuration; the cache works exact-match or semantic. Routing by prompt complexity and a visual flow editor are not documented on the pages reviewed.

The innFactory AI Gateway ships nine guardrail rule types - secret detection, PII block or redact, banned keywords, token and cost limit, Azure Content Safety, prompt injection, denied topics, groundedness and webhook - with shadow mode, mid-stream redaction and a GDPR preset. The MCP proxy exposes servers under /mcp/servers/{name}, with eight upstream authentication modes (including per-user PKCE, Entra on-behalf-of and AWS SigV4), toolsets, semantic tool search and guardrails on arguments and results. Added to this are an auto-router by prompt complexity, deployment groups and a Flow Builder with nine strategies, guardrail nodes with pass/block/fail branching and event triggers to webhooks; a flow is callable like a model.

What TrueFoundry does better

  • Deployment topologies: seven documented scenarios with sizing tables, Terraform modules for AWS, Azure and Google Cloud, Helm charts.
  • Identity depth: SAML 2.0, OIDC and SCIM, resource-specific role sets and virtual accounts.
  • Model breadth: considerably more connected models and platforms, including Groq, Cohere, Databricks and OpenRouter.
  • Guardrail ecosystem: Bedrock Guardrails, Google Model Armor, OPA, Cedar plus partners such as Enkrypt AI, Prisma AIRS and Akto.
  • MCP convenience, cache, observability: pre-built MCP servers, OpenAPI-to-MCP conversion, semantic cache, OpenTelemetry with integrations for Datadog, Coralogix, Last9 and Patronus.
  • Platform scope and attestations: Agent Gateway and AI Deployment under the same control plane; vendor-stated SOC 2, ISO 27001, HIPAA and ITAR compliance and, by its own account, recognition in the 2025 Gartner Market Guide.

When TrueFoundry is the right choice

  • You already run Kubernetes on AWS, Azure or Google Cloud and want gateway, control plane and optionally model deployment from a single source.
  • You need SAML or SCIM, resource-specific roles and virtual accounts.
  • You want to connect guardrails from Bedrock, Google Model Armor, OPA or Cedar.
  • A vendor-operated login and licensing server is uncritical for you, or you plan the Enterprise plan with direct IdP connection.

When the innFactory AI Gateway is the right choice

  • Gateway, dashboard and data should live in your own tenant, with login exclusively through your Entra ID or Keycloak.
  • You want to see agent time and LLM time of Claude Code, Codex and OpenCode per repository and cost center - including usage via Claude Max or a Codex plan.
  • Cost centers with hard stops per member and euro display at the ECB rate are decisive for your internal chargeback.
  • You rely on STACKIT, IONOS or Vertex with the EU endpoint and need native Anthropic or Gemini dialects or realtime WebSockets.
  • You use CompanyGPT and want to maintain identity, budgets and guardrails once.

Comparison at a glance

CriterionTrueFoundry AI Gateway (according to documentation, as of 26 Sep 2026)innFactory AI Gateway
Source codeGateway core not published as open source according to public repositories; Terraform modules and Cognita Apache-2.0, Helm chart repo and SDKs public without a declared licenseProprietary, Rust binary plus Next.js dashboard
Operating modelSeven scenarios from SaaS to fully self-hosted, all on Kubernetes; LLM path in own infrastructure with own gateway planeContainer in the customer tenant (Azure Container Apps/AKS, STACKIT, Kubernetes); no SaaS
Login and licenseGlobal authentication and licensing server with the vendor (e-mails, request count); direct IdP on-prem on the Enterprise planLogin against the customer’s Entra ID or OIDC; monthly per-user-tier license
SSO and rolesOIDC, SAML 2.0, SCIM; resource-specific roles; virtual accountsEntra ID, OIDC (Keycloak, Cognito); seven roles in every tier; no SAML/SCIM
BudgetsRules by users, teams, virtual accounts, models, metadata; block or audit mode; currency not documentedGates at organisation, cost center (pool/per-user), user, key; HTTP 402/E4006; fallback model; euros at ECB rate
Coding agentsGuides for Claude Code, Cline, OpenCode; change announcement on Gemini CLI; measurement of gateway requests; companion not describedSix proxy setups; companion for agent time, LLM time and subscription usage; MDM rollout
Providers and dialects1000+ or 1600+ models depending on the page; OpenAI-compatible schema; STACKIT/IONOS not documented13 provider types incl. STACKIT, IONOS; OpenAI, Anthropic, Gemini dialects; realtime
Guardrails and MCPValidate/Mutate; Audit, Enforce; Bedrock, Model Armor, OPA, Cedar; MCP registry with Virtual MCP ServersNine rule types; shadow mode; MCP proxy with eight auth modes and toolsets
RoutingWeight-, latency-, priority-based; Virtual Models; semantic cache; visual flow editor not documentedAuto-router by complexity; Flow Builder with nine strategies and guardrail/webhook nodes

Conclusion

TrueFoundry is a mature Kubernetes platform for enterprises that want gateway, control plane, MCP gateway and optionally model deployment from a single source. Its deployment documentation states clearly which component stays with the vendor and which data flows there.

The innFactory AI Gateway answers a narrower question: a component in your own tenant, with login against your own identity provider, cost centers with hard stops in euros and a companion that makes the working time of coding agents visible - even where developers work on their subscription. If those questions come first for you, you will find the better fit with us; if you are looking for a platform, TrueFoundry deserves serious consideration.

Natively integrated with CompanyGPT, one reusable stack

The innFactory AI Gateway is natively integrated with CompanyGPT: the same budgets, cost centers and guardrails apply to chat, agents and add-ons. Identity via Entra ID or Keycloak, cost centers, guardrails and the MCP registry form one stack that is reused across CompanyGPT, the gateway and your own tools. The gateway also runs standalone without CompanyGPT. You will find an overview of all comparisons under all comparisons and the product page under AI Gateway.

Sources

Retrieved on 26 September 2026:

  • TrueFoundry homepage with footer (Ensemble Labs Inc) — https://www.truefoundry.com/
  • TrueFoundry llms.txt (product description, financing, coding agents, MCP Gateway, compliance, analyst mention) — https://www.truefoundry.com/llms.txt
  • AI Gateway product page — https://www.truefoundry.com/ai-gateway
  • Intro to LLM Gateway — https://www.truefoundry.com/docs/gateway/intro-to-llm-gateway
  • Gateway overview (OpenAI-compatible schema, OpenTelemetry) — https://www.truefoundry.com/docs/gateway
  • Deployment Overview (seven scenarios, data flows, compliance statements) — https://www.truefoundry.com/docs/platform/deployment-overview
  • Gateway Plane Architecture (Global Authentication/Licensing Server) — https://www.truefoundry.com/docs/platform/gateway-plane-architecture
  • Deploy Control Plane and Gateway Plane (license key, image pull secret, egress, sizing) — https://www.truefoundry.com/docs/platform/deploy-control-plane-and-gateway-plane
  • SSO Overview (OIDC, SAML, SCIM, auth server options) — https://www.truefoundry.com/docs/platform/sso/overview
  • Manage User Roles and Permissions — https://www.truefoundry.com/docs/platform/manage-user-roles-and-permissions
  • Virtual Account Management — https://www.truefoundry.com/docs/platform/virtual-account-management
  • Budget Limiting — https://www.truefoundry.com/docs/ai-gateway/budgetlimiting
  • Rate Limiting — https://www.truefoundry.com/docs/ai-gateway/ratelimiting
  • Supported Providers — https://www.truefoundry.com/docs/ai-gateway/supported-providers
  • OpenRouter integration — https://www.truefoundry.com/docs/ai-gateway/openrouter
  • Virtual Models — https://www.truefoundry.com/docs/ai-gateway/virtual-model
  • Load Balancing Overview — https://www.truefoundry.com/docs/ai-gateway/load-balancing-overview
  • Caching — https://www.truefoundry.com/docs/ai-gateway/caching
  • Guardrails Overview — https://www.truefoundry.com/docs/ai-gateway/guardrails-overview
  • Guardrail integrations — https://www.truefoundry.com/docs/ai-gateway/bedrock-guardrails, https://www.truefoundry.com/docs/ai-gateway/google-model-armor, https://www.truefoundry.com/docs/ai-gateway/opa-guardrails, https://www.truefoundry.com/docs/ai-gateway/cedar-guardrails
  • MCP Gateway Auth and Security — https://www.truefoundry.com/docs/ai-gateway/mcp/mcp-gateway-auth-security
  • Blog: Enterprise Security for Claude — https://www.truefoundry.com/blog/enterprise-security-for-claude
  • Blog: Cline with TrueFoundry AI Gateway — https://www.truefoundry.com/blog/cline-integration-with-truefoundry-ai-gateway
  • Blog: OpenCode in-house — https://www.truefoundry.com/blog/bringing-opencode-in-house-secure-tool-usage-on-truefoundry
  • Change announcement: Gemini CLI model registration — https://www.truefoundry.com/docs/change-announcements/gemini-cli-model-registration-v0
  • Blog: integrations Coralogix, Last9, Patronus — https://www.truefoundry.com/blog/coralogix-integration-with-truefoundry-ai-gateway, https://www.truefoundry.com/blog/truefoundry-ai-gateway-integration-with-last9, https://www.truefoundry.com/blog/patronus-integration-with-truefoundrys-ai-gateway
  • GitHub organisation TrueFoundry (infra-charts, Terraform modules, SDKs, Cognita, models, tfy-claude-auth-token) — https://github.com/truefoundry

Note on the information: All statements about other vendors’ products are based on their publicly available documentation as of the stated date. Vendors continuously develop their products, features and terms — the vendor’s current information always prevails. If any statement appears outdated or incorrect to you, please drop us a line at info@innfactory.ai; we will review and correct it promptly. This comparison does not replace legal or data-protection advice in individual cases.

Further reading

Tobias Jonas
Written by

Tobias Jonas

Co-CEO, M.Sc.

Tobias Jonas, M.Sc. ist Mitgründer und Co-CEO der innFactory AI Consulting GmbH. Er ist ein führender Innovator im Bereich Künstliche Intelligenz und Cloud Computing. Als Co-Founder der innFactory GmbH hat er hunderte KI- und Cloud-Projekte erfolgreich geleitet und das Unternehmen als wichtigen Akteur im deutschen IT-Sektor etabliert. Dabei ist Tobias immer am Puls der Zeit: Er erkannte früh das Potenzial von KI Agenten und veranstaltete dazu eines der ersten Meetups in Deutschland. Zudem wies er bereits im ersten Monat nach Veröffentlichung auf das MCP Protokoll hin und informierte seine Follower am Gründungstag über die Agentic AI Foundation. Neben seinen Geschäftsführerrollen engagiert sich Tobias Jonas in verschiedenen Fach- und Wirtschaftsverbänden, darunter der KI Bundesverband und der Digitalausschuss der IHK München und Oberbayern, und leitet praxisorientierte KI- und Cloudprojekte an der Technischen Hochschule Rosenheim. Als Keynote Speaker teilt er seine Expertise zu KI und vermittelt komplexe technologische Konzepte verständlich.

LinkedIn