Getting started
Overview
BaseLLM is an Enterprise AI Control Plane that runs on your own infrastructure and routes all of your organization's AI traffic through one gateway. You manage the LLM Gateway, usage limits, budgets, sensitive data scanning and the Audit Log from a single dashboard.
Start here
This page summarizes what the product does. Installation and integration details are on the pages below.
- How it works
Getting started: The steps a request goes through in the gateway, and the records it writes - Installation options
Getting started: Docker, Helm, the setup wizard, licensing and sizing - Connect your app
Getting started: Project API key, the X-FFM-User header and error codes - OpenAI-compatible endpoint
Integration: Streaming, tool calling, response headers and limits - Claude Code and the Anthropic SDK
Integration: Anthropic Messages format, connecting Claude Code - Claude Code privacy notice
Integration: A sample notice for employees
Architecture
BaseLLM is built in three layers: traffic (LLM Gateway), process (Workflow Orchestration) and control (Enterprise AI Governance). The modules below make up the traffic layer and the controls that sit on top of it.
- LLM Gateway
Traffic: OpenAI-compatible chat endpoint; fallback model set per project, off by default - Policy Engine
Limits: Limits at the project, API key, user and model level - Sensitive data protection
Scanning: Secret scanning is on in every project; personal data masking is enabled by license - Cost Observability
Cost: Monthly project budget in USD, cost trend and month-end forecast - Audit Log
Audit: Tamper-proof record chain and batch export to your SIEM - Caching and memory
Efficiency: Provider and response caching; session, daily and project memory
How a request flows
Your application sends requests to BaseLLM's OpenAI-compatible endpoint, not to the model provider. Every check runs before the model is called, so you don't pay the provider for a rejected request.
- The project is identified from the API key, and the user from the X-FFM-User header.
- Project, API key and user limits are checked.
- The first model in the priority order that still has limit available is selected.
- The request is scanned for secrets and personal data.
- The model is called. If fallback is on and the provider returns an error, the request moves to the fallback model.
- Tokens, cost and the audit log entry are linked by the same correlation_id.
Key figures
| Capability | Value | Conditions |
|---|---|---|
| Multiple providers | OpenAI, Anthropic, Gemini, OpenRouter, in-house models | — |
| Added gateway latency | p99 1.73–1.76 ms | Test environment; excludes model time |
| Project limits | Limit of 10 with 50 concurrent requests: 10 passed | Redis on, multiple replicas |
| Personal data detection | 20 of 22 realistic samples caught | Full names and addresses out of scope |
| Audit chain verification | 244,939 records verified in 6.4 s | Local load test |