Who’s using AI, and how much? The answer, on one screen.
Stop asking every team separately. All AI traffic goes through one gateway: you write a policy once, it runs on every request, and every decision is logged.
- Runs on your own infrastructure: Docker or Kubernetes / OpenShift
- Switch providers without rewriting your apps
- Only the endpoint URL changes in your apps
Why BaseLLM: What teams ask before deploying.
BaseLLM is the control layer that manages model traffic from every chatbot, agent and LLM app in your organization, in one place. The only thing that changes in your apps is the endpoint they connect to.
- One control center for all AI usage
- Infrastructure that stays put when your LLM provider changes
- Costs you can see before the invoice arrives
- Automatic policies and an audit trail that's always ready
- One place to manage tool and MCP access
- Human-approved workflows
- An enterprise platform that runs on your own infrastructure
Setup and integration
Do we have to rewrite our apps?
No. Only the endpoint they connect to changes. BaseLLM exposes an OpenAI-compatible endpoint. Your existing chatbot, RAG and agent projects stay as they are: BaseLLM sits in front of them as a gateway and alongside them as a monitoring and governance layer.
Where does it run?
Today, on your own infrastructure, with Docker or Helm. Install it on a single server with Docker, or on Kubernetes and OpenShift with the Helm chart. On first launch, a browser-based setup wizard configures the license, certificates and connections. The four deployment models shown here are coming soon.
Does it integrate with our enterprise systems?
Yes: with your identity, logging and security infrastructure. Available today: LDAP sign-in, encrypted configuration, secrets management, multi-replica deployment with Helm, SIEM export, local embeddings (e5) and Qdrant / pgvector connections. Coming soon: SSO (SAML / OIDC) and fine-grained role permissions.
Does it work with the frameworks we already use?
It connects in front of, alongside or on top of what you've already built. Today you connect through the OpenAI-compatible endpoint. Ready-made integrations for LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel and Haystack are coming soon. BaseLLM can already be used as an MCP server.
How is the license verified?
A signed license file is verified in the product. The setup wizard generates a fingerprint that contains no company data. The portal signs a license for that specific installation, and the product checks the signature every time it starts. Copied, edited or expired licenses are rejected.
What happens to our data when the license ends?
Your data stays with you. Records are stored on your own infrastructure. When the license ends, LLM requests, workflows, scheduled reports, integrations and MCP stop. The dashboard stays open, and you can still view and export past records.
Security and data
Does our data leave the organization?
Only to the model provider you use. If you use an external provider, the text sent to the model goes to that provider, and BaseLLM scans it before it leaves. User identities are never passed to the provider, and personal data is never stored in raw form.
How is sensitive data protected before it reaches the model?
It's detected automatically and handled by policy. Turkish national ID numbers (TCKN), phone numbers, email addresses, IBANs, credit card numbers, tax IDs, passport numbers, license plates, IP addresses, dates of birth and customer numbers are detected automatically, and you can extend coverage with your own patterns. Depending on the project's policy, the data is masked, swapped for a placeholder, redacted with a type label, blocked, or only reported. Every detection is written to the Audit Log.
Can agents act on critical systems on their own?
Today, tool calling is switched on or off per project. In Tool Registry each tool has a risk level (read, write, destructive) and a list of allowed projects; a call that needs approval waits for a person.
What can we show an auditor?
The records are ready, in one place. Policy decisions, limit violations, personal data detections, configuration changes and request logs live in the Audit Log, with tokens and cost in the usage log under the same correlation ID. The log chain is tamper-proof and exported to your SIEM.
Does it replace KVKK compliance certification?
No. It supports the controls that KVKK (Turkey's data protection law) compliance requires. BaseLLM provides capabilities that support the controls sector regulations call for, but it is not a substitute for a compliance document or certification. Sensitive data use stays under control through PII detection, masking, usage policies and a tamper-proof audit log.
Providers
Are we locked into one model provider?
No. Your system stays the same even when the provider changes. A provider-agnostic adapter layer normalizes chat formats, tool-calling structures, streaming behavior, token data and error formats into one shape. Switching providers doesn't mean rebuilding your apps.
What happens if a provider fails?
If fallback is on, the request moves to the next model. When fallback models are enabled in the project settings and a provider returns an error or stops responding, the request moves to the next model in line, and your app never sees the error. Invalid requests don't fall back, and neither do streams that have already started. Every switch is recorded in the decision list and the audit log.
Will it keep up as usage grows?
The gateway runs as multiple replicas, while management stays in one place. In testing, a single replica handled ~400 requests per second and three replicas handled 648 (1 core and 1 GB per replica). In multi-replica deployments, limits are counted together through Redis. Real-world capacity is usually set by your provider's rate and quota limits.
Cost
Which team spends how much, and when do we find out?
As soon as the spend happens, broken down by project. Every model call is metered. Spend is visible by project and model, and token use is tracked per user, so you don't have to wait for the end-of-month invoice.
What happens if a budget is exceeded?
First a warning, then a block or just a log entry. When spend reaches the threshold (80% by default), a warning record goes to the Audit Log and your SIEM. Once the budget runs out, the project setting decides: the request is blocked with 429 budget_exceeded, or it goes through and is only logged.
How does cost come down?
The model sees only what it needs. Today: repeated prompt prefixes are read from the provider's cache at a discount, identical requests can be served from the response cache, long conversations are summarized, and simple questions are routed to a cheaper model. Long tool output is shortened before it reaches the model. Early stopping is coming soon. Savings are measured and reported.
How will we see a return on the investment?
AI risk you can't see today becomes visible. Beyond cutting cost, BaseLLM makes the AI risk you can't see today measurable.
Agents and workflows
How are workflows built?
Fixed steps, with an agent where needed. A workflow combines LLM calls, tool calls, conditions, agent steps and human approval. It starts manually, on a schedule or from a webhook. The same steps run every time, so results are predictable, cost stays under control and every run is audit-ready.
What if an agent gets stuck in a loop?
If the loop runs too long, it is stopped. An agent runs for at most 10 turns (5 by default) and calls at most 5 tools per turn. On the last turn tools are switched off and it must answer.
How does it remember company knowledge?
With managed memory in three layers. Session holds the active conversation automatically. Daily collects the day's summary and decisions semi-automatically. Project keeps goals, business rules and approved templates for the long term, and grows only with approval.
What happens at a risky step?
A person makes the call. When the Policy Engine decides approval is needed, the task goes to the project approvers. The approver reviews the model output, the tool call and the risk, then approves or rejects it.
Teams
Do our teams need technical skills?
No. Approved workflows are defined centrally. Finance, legal, HR, sales, operations and customer support teams use approved AI workflows without dealing with technical details.
What can we use it for?
One platform, many processes. No new use case starts from scratch: each one runs on the same LLM Gateway, Policy Engine and Audit Log.
Wouldn't banning AI be safer?
The goal isn't to ban AI. It's to bring it under control. BaseLLM doesn't ban AI. It makes your organization's AI use centrally managed, secure, measurable, auditable and easier to optimize. Teams keep using AI, and control stays at the center.
Documentation
- Overview
- How it works
- Installation options
- Connect your app
- Videos
- LLM Gateway
- Policy Engine
- Sensitive data masking
- Image and document reading (OCR)
- Prompt injection protection
- Budget and cost
- Audit Log
- Observability
- Reports
- Workflow
- Tool Registry
- Human approval
- MCP
- Memory
- OpenAI-compatible endpoint
- LangChain and other frameworks
- SSO / LDAP
- SIEM export
- Claude Code and the Anthropic SDK
- Integrations
- Employee privacy notice for Claude Code (sample)
- Frequently asked questions