Integration
OpenAI-compatible endpoint
BaseLLM exposes an endpoint in the OpenAI Chat API format: POST /api/v1/chat/completions. Streaming and tool calling are supported; tool calling is enabled per project.
Compatibility
Connection details, headers and error codes are covered in Connect your app. This page describes how BaseLLM handles request and response fields. Supported providers are OpenAI, Anthropic, Google (Gemini), OpenRouter and in-house models.
| Field | Behavior |
|---|---|
| user | Can be used instead of X-FFM-User; the header takes precedence |
| max_tokens | Lowered to the max output limit if it exceeds it; returns 200 |
| stop | Up to 4 values, each up to 256 characters; scanned |
| tools | Up to 128 tools; enabled per project with tools_enabled |
| tool_choice, parallel_tool_calls | Passed through to the provider |
| temperature | The response cache applies only when it is 0 |
| model | Ignored by default; the project picks the model. With client model choice on in the project, one of the project's own models or an allowed model name can be sent; empty or auto still lets the project pick. GET /api/v1/models lists the allowed models. With a model rule on the API key or the user (X-FFM-User), the list and the requests, auto included, narrow to it. A project with only provider connections (no models of its own) cannot answer auto; a model name is required |
Streaming
- Limit and policy errors are returned as HTTP errors before the stream starts.
- Fallback happens only before the first token is sent.
- If the provider fails after the stream has started, the client receives a stream_interrupted event.
- For streaming requests, token counts are always requested from the provider.
- Responses cut off midway are still logged; if no count arrives, ≈ 4 characters = 1 token is assumed.
Tool calling
The Chat API passes the tools, tool_choice and parallel_tool_calls fields through to the provider. The tool_calls field in the response has the same shape for OpenAI-compatible, Anthropic and Google models. The feature is enabled per project with tools_enabled and is off by default.
Response fields and headers
| Name | Type | Meaning |
|---|---|---|
| model | Response field | The model that was used; changes with routing |
| tool_calls | Response field | The model's tool calls; when tools_enabled is on |
| X-FFM-Cache | Response header | HIT, MISS, SHADOW or BYPASS |
| X-FFM-Memory | Response header | E.g. session=continued; summarized=14; kept=6 |
| scope | Error field | Where the limit was hit: project, api_key, user, llm |
Error responses include the code, message, retryAfterSeconds and scope fields. The response cache defaults to Shadow mode: the request still goes to the model, and the potential savings are logged.
Things to know
- If the request itself is invalid, there is no fallback. Fallback is off by default. Context window overflows and content policy refusals have their own project settings.
- The max output limit applies only to chat requests.
- In your application, you change the endpoint URL and the API key. For per-user limits, add X-FFM-User.