Getting started

How it works

Every request goes through BaseLLM between your application and the model provider. Limits and scans run before the model is called, so a rejected request costs nothing.

The path of a request

Checks run in the order below. A rejected request never reaches the model; the decision is still logged.

StepWhat happensIf it stops
1. IdentityProject from the API key, user from the header401
2. Prompt injectionScan for instructions aimed at the model (if on for the project)403
3. ScanSecret key and personal data scan403
4. Response cacheIf the cache is on and holds the answer, the request is answered here—
5. LimitsProject, API key and user limits429 or 400
6. Model choiceFirst model in priority order with limit available; with smart routing on, a simple question tries the cheaper model first429, scope: llm
7. Model callFallback model on provider error (if on)—
8. ResponseScanned if needed, returned to the app—
9. RecordTokens logged with key, user and model—

Why this order

  • Security comes first: the prompt injection and personal data scans run before limits, so a request stopped by a scan never reaches the limit check.
  • An answer from the cache calls no model and skips the limit check.
  • Model choice comes after the limits, so a request that will be stopped doesn't count against the model's counter.
  • All checks run before the model call, so a rejected request incurs no cost.
ResultACTIONWhat happens to the request
BlockedblockedDoesn't reach the model; the app gets an error
WarnedwarnedGoes through; only logged
ClampedclampedGoes through; response cut to the max output limit
ReroutedreroutedGoes to the next model; no error

Block or Warn only mode is chosen per project and applies to all four layers together. Policies are off by default. Setting changes apply at once; with more than one server they reach the others through Redis (within 5 s at most without Redis). Usage and budget counters update with a delay of up to 10 s.

Scanning, cache and memory

Secret key scanning is on for every license and every project, and blocks by default. Personal data masking comes with the license and is off by default.

  • Your app sends X-FFM-Session; if session memory is on, BaseLLM adds the history.
  • In long conversations, older messages are summarized; the last 6 messages (2–50, set per project) stay unchanged.
  • The response cache applies only to identical requests with temperature 0.
  • The response cache defaults to Shadow mode; the request still goes to the model.
  • The response cache and memory work only in projects with personal data protection on.
  • Raw personal data is never written to Redis; stored text is masked and encrypted.

Records

Token and cost data are kept in the usage record; audit events are kept in a separate PostgreSQL database. correlation_id links the audit, usage and log records of the same request. The prompt, model response, raw API key, provider credentials, request body and tool arguments are never written to the audit log.

SIEM export runs every 60 seconds by default; you can set the interval between 15 and 3600 seconds.