Getting started
How it works
Every request goes through BaseLLM between your application and the model provider. Limits and scans run before the model is called, so a rejected request costs nothing.
The path of a request
Checks run in the order below. A rejected request never reaches the model; the decision is still logged.
| Step | What happens | If it stops |
|---|---|---|
| 1. Identity | Project from the API key, user from the header | 401 |
| 2. Prompt injection | Scan for instructions aimed at the model (if on for the project) | 403 |
| 3. Scan | Secret key and personal data scan | 403 |
| 4. Response cache | If the cache is on and holds the answer, the request is answered here | — |
| 5. Limits | Project, API key and user limits | 429 or 400 |
| 6. Model choice | First model in priority order with limit available; with smart routing on, a simple question tries the cheaper model first | 429, scope: llm |
| 7. Model call | Fallback model on provider error (if on) | — |
| 8. Response | Scanned if needed, returned to the app | — |
| 9. Record | Tokens logged with key, user and model | — |
Why this order
- Security comes first: the prompt injection and personal data scans run before limits, so a request stopped by a scan never reaches the limit check.
- An answer from the cache calls no model and skips the limit check.
- Model choice comes after the limits, so a request that will be stopped doesn't count against the model's counter.
- All checks run before the model call, so a rejected request incurs no cost.
| Result | ACTION | What happens to the request |
|---|---|---|
| Blocked | blocked | Doesn't reach the model; the app gets an error |
| Warned | warned | Goes through; only logged |
| Clamped | clamped | Goes through; response cut to the max output limit |
| Rerouted | rerouted | Goes to the next model; no error |
Block or Warn only mode is chosen per project and applies to all four layers together. Policies are off by default. Setting changes apply at once; with more than one server they reach the others through Redis (within 5 s at most without Redis). Usage and budget counters update with a delay of up to 10 s.
Scanning, cache and memory
Secret key scanning is on for every license and every project, and blocks by default. Personal data masking comes with the license and is off by default.
- Your app sends X-FFM-Session; if session memory is on, BaseLLM adds the history.
- In long conversations, older messages are summarized; the last 6 messages (2–50, set per project) stay unchanged.
- The response cache applies only to identical requests with temperature 0.
- The response cache defaults to Shadow mode; the request still goes to the model.
- The response cache and memory work only in projects with personal data protection on.
- Raw personal data is never written to Redis; stored text is masked and encrypted.
Records
Token and cost data are kept in the usage record; audit events are kept in a separate PostgreSQL database. correlation_id links the audit, usage and log records of the same request. The prompt, model response, raw API key, provider credentials, request body and tool arguments are never written to the audit log.
SIEM export runs every 60 seconds by default; you can set the interval between 15 and 3600 seconds.