Getting started
Connect your app
Your application sends requests to BaseLLM's OpenAI-compatible chat endpoint with a project API key, instead of calling the model provider. If you use per-user limits, add the X-FFM-User header to every request.
Connection model
- Requests go to POST /api/v1/chat/completions; the server address is specific to your installation.
- Authentication uses a project API key; keys start with the sk-ffm prefix.
- The key identifies the project, not the person.
- Provider keys are stored in BaseLLM, not in your application.
- Policies are off by default; an admin turns on limits project by project.
Example request
from openai import OpenAI
client = OpenAI(
base_url="<basellm-endpoint>/api/v1",
api_key="sk-ffm-...", # project API key
)
resp = client.chat.completions.create(
model="basellm", # BaseLLM ignores this field
messages=[{"role": "user", "content": "Hello"}],
extra_headers={"X-FFM-User": "u-12345"},
)
Headers and fields
| Name | Type | Rule |
|---|---|---|
| X-FFM-User | Request header | End-user ID; up to 128 characters |
| X-FFM-User-Signature | Request header | When the project requires signed end-user IDs: t=<unix seconds>,v1=<hex HMAC-SHA256(signing secret, "t.id")>; valid for 5 minutes |
| user | Body field | Can replace the header; if both are sent, the header wins |
| X-FFM-Session | Request header | Session ID, 1–128 characters; for session memory |
| correlation_id | Record field | Links audit, usage and log records |
| X-FFM-Cache | Response header | Response cache: HIT, MISS, SHADOW, BYPASS |
| X-FFM-Memory | Response header | What memory did, e.g. session=continued; kept=6 |
X-FFM-User is not forwarded to the model provider; it is used for limits, model access and reporting. Your application verifies the identity; BaseLLM uses the value as received. Send a short internal code rather than a full name. If end users hold the project key, they could write someone else's ID; to prevent that, require signed end-user IDs in the project and sign the ID from your app, keeping the signing secret on your server only.
Error codes
| Condition | HTTP | Code | What your app should do |
|---|---|---|---|
| Per-minute request limit exceeded | 429 | rate_limited | Wait retryAfterSeconds, then retry |
| Daily or monthly token quota used up | 429 | quota_exceeded | Don't retry; notify an admin |
| Input larger than allowed | 400 | context_too_large | Shorten or split the text |
| User limit on, no user ID | 400 | user_required | Add a user ID to every request |
| All project models at limit | 429 | rate_limited / quota_exceeded | Try again later; scope: llm |
| Secret key detected | 403 | secret_blocked | Remove the key from the text and rotate it |
| Personal data blocked | 403 | pii_blocked | Remove the personal data from the text |
| The project has no models of its own, only provider connections, and no model was named | 400 | model_required | Put an allowed model name in model (GET /api/v1/models) |
| End-user ID unsigned or signature invalid | 401 | user_signature_invalid | Sign X-FFM-User-Signature with the current time and the right secret |
| Requested model not allowed, unpriced or over the cap | 400 | model_not_allowed / model_unpriced / model_over_price_cap | Pick one from GET /api/v1/models or set model to auto; scope: api_key or user means the key's or the user's rule |
| Project monthly budget used up | 429 | budget_exceeded | Notify an admin |
On limit errors, the scope field shows where the problem is: project, api_key, user or llm. The max output limit doesn't produce an error; the request completes with 200 and the response is truncated. Routing on a reached limit also returns 200; the model field in the response shows the model that was used.
Recommended rollout
On a new project, turn limits on in Warn only mode and watch the records for a week or two. "No user ID" records point to integrations that don't send an ID. If the values don't interrupt normal usage, switch to Block mode.
- Quotas are checked at the start of each request; with concurrent requests they can be exceeded by a few requests.
- Input size is an estimate (4 characters ≈ 1 token); Turkish text can use more tokens.
- Setting changes apply at once; with more than one server they reach the others through Redis (within 5 s at most without Redis). Usage and budget counters update with a delay of up to 10 s.
- The max output limit applies only to chat requests.