Integration

OpenAI-compatible endpoint

BaseLLM exposes an endpoint in the OpenAI Chat API format: POST /api/v1/chat/completions. Streaming and tool calling are supported; tool calling is enabled per project.

Compatibility

Connection details, headers and error codes are covered in Connect your app. This page describes how BaseLLM handles request and response fields. Supported providers are OpenAI, Anthropic, Google (Gemini), OpenRouter and in-house models.

FieldBehavior
userCan be used instead of X-FFM-User; the header takes precedence
max_tokensLowered to the max output limit if it exceeds it; returns 200
stopUp to 4 values, each up to 256 characters; scanned
toolsUp to 128 tools; enabled per project with tools_enabled
tool_choice, parallel_tool_callsPassed through to the provider
temperatureThe response cache applies only when it is 0
modelIgnored by default; the project picks the model. With client model choice on in the project, one of the project's own models or an allowed model name can be sent; empty or auto still lets the project pick. GET /api/v1/models lists the allowed models. With a model rule on the API key or the user (X-FFM-User), the list and the requests, auto included, narrow to it. A project with only provider connections (no models of its own) cannot answer auto; a model name is required

Streaming

  • Limit and policy errors are returned as HTTP errors before the stream starts.
  • Fallback happens only before the first token is sent.
  • If the provider fails after the stream has started, the client receives a stream_interrupted event.
  • For streaming requests, token counts are always requested from the provider.
  • Responses cut off midway are still logged; if no count arrives, ≈ 4 characters = 1 token is assumed.

Tool calling

The Chat API passes the tools, tool_choice and parallel_tool_calls fields through to the provider. The tool_calls field in the response has the same shape for OpenAI-compatible, Anthropic and Google models. The feature is enabled per project with tools_enabled and is off by default.

Response fields and headers

NameTypeMeaning
modelResponse fieldThe model that was used; changes with routing
tool_callsResponse fieldThe model's tool calls; when tools_enabled is on
X-FFM-CacheResponse headerHIT, MISS, SHADOW or BYPASS
X-FFM-MemoryResponse headerE.g. session=continued; summarized=14; kept=6
scopeError fieldWhere the limit was hit: project, api_key, user, llm

Error responses include the code, message, retryAfterSeconds and scope fields. The response cache defaults to Shadow mode: the request still goes to the model, and the potential savings are logged.

Things to know

  • If the request itself is invalid, there is no fallback. Fallback is off by default. Context window overflows and content policy refusals have their own project settings.
  • The max output limit applies only to chat requests.
  • In your application, you change the endpoint URL and the API key. For per-user limits, add X-FFM-User.