Getting started

Connect your app

Your application sends requests to BaseLLM's OpenAI-compatible chat endpoint with a project API key, instead of calling the model provider. If you use per-user limits, add the X-FFM-User header to every request.

Connection model

  • Requests go to POST /api/v1/chat/completions; the server address is specific to your installation.
  • Authentication uses a project API key; keys start with the sk-ffm prefix.
  • The key identifies the project, not the person.
  • Provider keys are stored in BaseLLM, not in your application.
  • Policies are off by default; an admin turns on limits project by project.

Example request

from openai import OpenAI
client = OpenAI(
    base_url="<basellm-endpoint>/api/v1",
    api_key="sk-ffm-...",  # project API key
)
resp = client.chat.completions.create(
    model="basellm",  # BaseLLM ignores this field
    messages=[{"role": "user", "content": "Hello"}],
    extra_headers={"X-FFM-User": "u-12345"},
)

Headers and fields

NameTypeRule
X-FFM-UserRequest headerEnd-user ID; up to 128 characters
X-FFM-User-SignatureRequest headerWhen the project requires signed end-user IDs: t=<unix seconds>,v1=<hex HMAC-SHA256(signing secret, "t.id")>; valid for 5 minutes
userBody fieldCan replace the header; if both are sent, the header wins
X-FFM-SessionRequest headerSession ID, 1–128 characters; for session memory
correlation_idRecord fieldLinks audit, usage and log records
X-FFM-CacheResponse headerResponse cache: HIT, MISS, SHADOW, BYPASS
X-FFM-MemoryResponse headerWhat memory did, e.g. session=continued; kept=6

X-FFM-User is not forwarded to the model provider; it is used for limits, model access and reporting. Your application verifies the identity; BaseLLM uses the value as received. Send a short internal code rather than a full name. If end users hold the project key, they could write someone else's ID; to prevent that, require signed end-user IDs in the project and sign the ID from your app, keeping the signing secret on your server only.

Error codes

ConditionHTTPCodeWhat your app should do
Per-minute request limit exceeded429rate_limitedWait retryAfterSeconds, then retry
Daily or monthly token quota used up429quota_exceededDon't retry; notify an admin
Input larger than allowed400context_too_largeShorten or split the text
User limit on, no user ID400user_requiredAdd a user ID to every request
All project models at limit429rate_limited / quota_exceededTry again later; scope: llm
Secret key detected403secret_blockedRemove the key from the text and rotate it
Personal data blocked403pii_blockedRemove the personal data from the text
The project has no models of its own, only provider connections, and no model was named400model_requiredPut an allowed model name in model (GET /api/v1/models)
End-user ID unsigned or signature invalid401user_signature_invalidSign X-FFM-User-Signature with the current time and the right secret
Requested model not allowed, unpriced or over the cap400model_not_allowed / model_unpriced / model_over_price_capPick one from GET /api/v1/models or set model to auto; scope: api_key or user means the key's or the user's rule
Project monthly budget used up429budget_exceededNotify an admin

On limit errors, the scope field shows where the problem is: project, api_key, user or llm. The max output limit doesn't produce an error; the request completes with 200 and the response is truncated. Routing on a reached limit also returns 200; the model field in the response shows the model that was used.

On a new project, turn limits on in Warn only mode and watch the records for a week or two. "No user ID" records point to integrations that don't send an ID. If the values don't interrupt normal usage, switch to Block mode.

  • Quotas are checked at the start of each request; with concurrent requests they can be exceeded by a few requests.
  • Input size is an estimate (4 characters ≈ 1 token); Turkish text can use more tokens.
  • Setting changes apply at once; with more than one server they reach the others through Redis (within 5 s at most without Redis). Usage and budget counters update with a delay of up to 10 s.
  • The max output limit applies only to chat requests.