Control
Policy Engine
Policy Engine checks every request against project, API key, user and model limits before the model is called. A rejected request costs nothing at the provider, and every decision is logged.
Policies
Policies are off by default; an admin turns them on per project. Each project has its own limits, so one project going over never affects another. A limit left empty means "unlimited".
| Policy | What it limits | Result in Block mode |
|---|---|---|
| rate_limit | Requests per minute | 429 rate_limited |
| daily_token_quota | Total tokens per day | 429 quota_exceeded |
| monthly_token_quota | Total tokens per month | 429 quota_exceeded |
| max_input_tokens | Text sent to the model in one request | 400 context_too_large |
| max_output_tokens | Maximum tokens in one response | Request passes, response is clamped |
| user_required | Requests without an identity while user limits are on | 400 user_required |
| monthly_budget, budget_alert | Project monthly USD budget and alert threshold | 429 budget_exceeded |
Budget policies are covered in detail on the Budget and cost page.
Windows and modes
- The per-minute limit uses a sliding window: each request looks at the last 60 seconds.
- The limit applies to the total across all of a project's API keys.
- A rejected request includes retryAfterSeconds, which tells you when to retry.
- Quotas count the input + output tokens reported by the provider.
- Days and months start according to the USAGE_QUOTA_TIMEZONE setting (Europe/Istanbul).
- Input size is estimated before sending: roughly 4 characters = 1 token.
| Situation | Block | Warn only |
|---|---|---|
| Per-minute limit or quota exceeded | Rejected with 429 | Request passes, is logged |
| Input too large | Rejected with 400 | Request passes |
| Output limit exceeded | max_tokens is lowered to the limit | Not clamped |
| Request without identity (user limits on) | 400 user_required | Passes, is logged |
| Model limit reached | Rerouted or 429 | Request goes to the first model |
The enforcement mode applies to every layer of the project. Recommended rollout: turn limits on in "Warn only" mode, watch them for a week or two, then switch to "Block". Quotas are checked when a request starts, so concurrent requests can push usage over by a few requests.
Limit layers and user identity
| Layer | Scope | Note |
|---|---|---|
| Project | Total rate and usage for the project | Input/output limits only at this layer |
| API key | Each key has its own counter | One integration can't affect another |
| User | Each end user has their own counter | Requires X-FFM-User |
| Model | Each model on the project has its own counter | When full, block or move to the next |
- Every layer has per-minute request, daily token and monthly token policies.
- The identity comes from the X-FFM-User header or the user field in the chat request body.
- If both are sent, the header wins; an identity can be at most 128 characters.
- The identity is never forwarded to the model provider; it is used only for limits and reporting.
- Your application verifies the identity; BaseLLM uses the value it receives.
- You set a default limit and exceptions (custom limits) for keys and users.
When a model limit is reached, the default behavior is "Block request"; optionally, the request is routed to the next model in priority order. If no model has capacity, the request is rejected with 429 (rate_limited or quota_exceeded, scope: llm). A model skipped because it was full doesn't count that request against its own counter.
Error codes and outcomes
| Situation | HTTP | Code | What your app should do |
|---|---|---|---|
| Per-minute limit exceeded | 429 | rate_limited | Wait retryAfterSeconds, then retry |
| Daily or monthly quota used up | 429 | quota_exceeded | Don't retry; notify an admin |
| Input larger than allowed | 400 | context_too_large | Shorten or split the text |
| Request without identity | 400 | user_required | Add a user identity to every request |
| All of the project's models are full | 429 | rate_limited / quota_exceeded | Try again later |
| Project monthly budget used up | 429 | budget_exceeded | Notify an admin |
| Output clamped or rerouted | 200 | No error | Nothing to do |
| ACTION | On screen | What happens to the request |
|---|---|---|
| blocked | Blocked | Never reaches the model; the app gets an error |
| warned | Warning | Passes; only logged |
| clamped | Clamped | Passes; the response is cut to the limit |
| rerouted | Rerouted | Goes to the next model; no error |
| failed | — | Provider error and no fallback model left |
The scope field in limit errors tells you where the problem is: project, api_key, user or llm. For streaming requests, the error comes back as a proper HTTP error before the stream starts.
Logging and multiple instances
| Topic | Behavior |
|---|---|
| Suppression | 1 record per minute for the same policy + suppressedSinceLast |
| Audit | Every decision also goes to the Audit Log as policy.violation |
| Retention | 90 days by default; POLICY_DECISION_RETENTION_DAYS |
| Propagation | Setting changes apply at once; with more than one server they reach the others through Redis (within 5 s at most without Redis). Usage and budget counters update with a delay of up to 10 s. |
| Redis | Per-minute counter; required with multiple replicas (REDIS_URL) |
| If Redis is unreachable | Traffic keeps flowing; each replica temporarily falls back to its own counter |