Getting started

Overview

BaseLLM is an Enterprise AI Control Plane that runs on your own infrastructure and routes all of your organization's AI traffic through one gateway. You manage the LLM Gateway, usage limits, budgets, sensitive data scanning and the Audit Log from a single dashboard.

Start here

This page summarizes what the product does. Installation and integration details are on the pages below.

Architecture

BaseLLM is built in three layers: traffic (LLM Gateway), process (Workflow Orchestration) and control (Enterprise AI Governance). The modules below make up the traffic layer and the controls that sit on top of it.

  • LLM Gateway Traffic: OpenAI-compatible chat endpoint; fallback model set per project, off by default
  • Policy Engine Limits: Limits at the project, API key, user and model level
  • Sensitive data protection Scanning: Secret scanning is on in every project; personal data masking is enabled by license
  • Cost Observability Cost: Monthly project budget in USD, cost trend and month-end forecast
  • Audit Log Audit: Tamper-proof record chain and batch export to your SIEM
  • Caching and memory Efficiency: Provider and response caching; session, daily and project memory

How a request flows

Your application sends requests to BaseLLM's OpenAI-compatible endpoint, not to the model provider. Every check runs before the model is called, so you don't pay the provider for a rejected request.

  • The project is identified from the API key, and the user from the X-FFM-User header.
  • Project, API key and user limits are checked.
  • The first model in the priority order that still has limit available is selected.
  • The request is scanned for secrets and personal data.
  • The model is called. If fallback is on and the provider returns an error, the request moves to the fallback model.
  • Tokens, cost and the audit log entry are linked by the same correlation_id.

Key figures

CapabilityValueConditions
Multiple providersOpenAI, Anthropic, Gemini, OpenRouter, in-house models—
Added gateway latencyp99 1.73–1.76 msTest environment; excludes model time
Project limitsLimit of 10 with 50 concurrent requests: 10 passedRedis on, multiple replicas
Personal data detection20 of 22 realistic samples caughtFull names and addresses out of scope
Audit chain verification244,939 records verified in 6.4 sLocal load test