Control

Policy Engine

Policy Engine checks every request against project, API key, user and model limits before the model is called. A rejected request costs nothing at the provider, and every decision is logged.

Policies

Policies are off by default; an admin turns them on per project. Each project has its own limits, so one project going over never affects another. A limit left empty means "unlimited".

PolicyWhat it limitsResult in Block mode
rate_limitRequests per minute429 rate_limited
daily_token_quotaTotal tokens per day429 quota_exceeded
monthly_token_quotaTotal tokens per month429 quota_exceeded
max_input_tokensText sent to the model in one request400 context_too_large
max_output_tokensMaximum tokens in one responseRequest passes, response is clamped
user_requiredRequests without an identity while user limits are on400 user_required
monthly_budget, budget_alertProject monthly USD budget and alert threshold429 budget_exceeded

Budget policies are covered in detail on the Budget and cost page.

Windows and modes

  • The per-minute limit uses a sliding window: each request looks at the last 60 seconds.
  • The limit applies to the total across all of a project's API keys.
  • A rejected request includes retryAfterSeconds, which tells you when to retry.
  • Quotas count the input + output tokens reported by the provider.
  • Days and months start according to the USAGE_QUOTA_TIMEZONE setting (Europe/Istanbul).
  • Input size is estimated before sending: roughly 4 characters = 1 token.
SituationBlockWarn only
Per-minute limit or quota exceededRejected with 429Request passes, is logged
Input too largeRejected with 400Request passes
Output limit exceededmax_tokens is lowered to the limitNot clamped
Request without identity (user limits on)400 user_requiredPasses, is logged
Model limit reachedRerouted or 429Request goes to the first model

The enforcement mode applies to every layer of the project. Recommended rollout: turn limits on in "Warn only" mode, watch them for a week or two, then switch to "Block". Quotas are checked when a request starts, so concurrent requests can push usage over by a few requests.

Limit layers and user identity

LayerScopeNote
ProjectTotal rate and usage for the projectInput/output limits only at this layer
API keyEach key has its own counterOne integration can't affect another
UserEach end user has their own counterRequires X-FFM-User
ModelEach model on the project has its own counterWhen full, block or move to the next
  • Every layer has per-minute request, daily token and monthly token policies.
  • The identity comes from the X-FFM-User header or the user field in the chat request body.
  • If both are sent, the header wins; an identity can be at most 128 characters.
  • The identity is never forwarded to the model provider; it is used only for limits and reporting.
  • Your application verifies the identity; BaseLLM uses the value it receives.
  • You set a default limit and exceptions (custom limits) for keys and users.

When a model limit is reached, the default behavior is "Block request"; optionally, the request is routed to the next model in priority order. If no model has capacity, the request is rejected with 429 (rate_limited or quota_exceeded, scope: llm). A model skipped because it was full doesn't count that request against its own counter.

Error codes and outcomes

SituationHTTPCodeWhat your app should do
Per-minute limit exceeded429rate_limitedWait retryAfterSeconds, then retry
Daily or monthly quota used up429quota_exceededDon't retry; notify an admin
Input larger than allowed400context_too_largeShorten or split the text
Request without identity400user_requiredAdd a user identity to every request
All of the project's models are full429rate_limited / quota_exceededTry again later
Project monthly budget used up429budget_exceededNotify an admin
Output clamped or rerouted200No errorNothing to do
ACTIONOn screenWhat happens to the request
blockedBlockedNever reaches the model; the app gets an error
warnedWarningPasses; only logged
clampedClampedPasses; the response is cut to the limit
reroutedReroutedGoes to the next model; no error
failed—Provider error and no fallback model left

The scope field in limit errors tells you where the problem is: project, api_key, user or llm. For streaming requests, the error comes back as a proper HTTP error before the stream starts.

Logging and multiple instances

TopicBehavior
Suppression1 record per minute for the same policy + suppressedSinceLast
AuditEvery decision also goes to the Audit Log as policy.violation
Retention90 days by default; POLICY_DECISION_RETENTION_DAYS
PropagationSetting changes apply at once; with more than one server they reach the others through Redis (within 5 s at most without Redis). Usage and budget counters update with a delay of up to 10 s.
RedisPer-minute counter; required with multiple replicas (REDIS_URL)
If Redis is unreachableTraffic keeps flowing; each replica temporarily falls back to its own counter