Getting started

Installation options

BaseLLM is the on-prem AI gateway by Base Interactive that runs in your own infrastructure. You install it with Docker on a single server, or with a Helm chart on Kubernetes and OpenShift. On first start, a browser-based setup wizard walks you through the license, certificate and connections step by step.

Installation paths

PathWhen to use
Docker, single serverPilots and small installations
Helm: Kubernetes / OpenShiftMultiple replicas, high availability

With Helm, PostgreSQL runs on your own server; the Docker package ships its own PostgreSQL. Either way you need two databases, one main and one for audit. Redis is required with more than one replica and not needed on a single server. See the Sizing section for starting hardware recommendations.

The Docker installation needs Docker Engine 20.10 and Docker Compose 2.20 or later. The package runs on Linux, macOS and Windows. On Windows, run the installation commands in the Ubuntu terminal of WSL2; Command Prompt (cmd) and PowerShell do not have the commands the guide uses. The steps are in KURULUM.md inside the package.

Kubernetes and OpenShift

helm install basellm helm/<chart-from-package>.tgz -n basellm \
  --set hosts.ui=basellm.company.local \
  --set hosts.api=basellm-api.company.local \
  --set platform=openshift
  • Backend (API) and UI
  • Ingress / OpenShift Route
  • Optional Redis (shared limit counter)
  • Persistent volume for the setup wizard
  • One-time database migration job
  • Kubernetes Secret: encryption key and settings
  • Service account that reads the cluster's identity (read access to the kube-system namespace only)

Setup wizard, license and encrypted configuration

StepNameWhat it sets up
1Language & regionSystem language and the default personal data country for new projects
2LicenseYou give the fingerprint to the portal and receive a signed license; the license is tied to this server or cluster
3CertificateThe certificate that locks the encryption key
4DatabaseMain database (PostgreSQL)
5Audit databaseSeparate database for audit records; can be on the same server as the main one
6RedisShared limit counters; not needed on a single server
7LDAPDirectory connection; optional, can also be set later in Settings
8AdminFirst admin account
9ConfirmSummary of the settings entered; setup finishes here

At each step, you test the connection with "Test" before it is saved. Once the wizard finishes, it can't be run again. Secret values sit on disk encrypted with the enc:v1: prefix and are decrypted only at startup, in memory. The key is locked with your certificate; config_cli provides the init, encrypt-env, encrypt-value, rewrap and verify commands.

Which server is the license tied to?

The enterprise license is tied to an identity that does not move with the database. So even if the database is copied to another server, the license doesn't work there.

InstallationLicense is tied to
DockerThe server's machine ID (/etc/machine-id). If the file is missing, create it with sudo systemd-machine-id-setup
Kubernetes / OpenShiftThe cluster's identity. Scaling replicas or adding and removing servers doesn't affect it

To read the cluster's identity, the Helm chart creates a service account that may only read the kube-system namespace. If your organization prefers to grant this with its own account, use rbac.create=false and backend.serviceAccountName; without the permission, the application doesn't start and says why.

When you move to another server, or to a new or standby cluster, give the new installation's fingerprint to the portal and get a new license.

Sizing and high availability

The values below are starting recommendations; confirm actual capacity with a load test in your environment. In practice, the provider's rate and quota limits usually set your capacity first.

ComponentStarting recommendation
BaseLLMAt least 2 replicas, each 1 core + 1 GB
Replica capacity~400 non-streaming or ~100–150 streaming requests per second
PostgreSQL~0.1 core per 100 requests per second; fast disk
RedisRequired with multiple replicas; negligible load
Load balancerFailover to another replica on errors and 503 enabled
  • Multiple replicas: if one backend goes down, another handles the requests.
  • At least one replica stays up during node maintenance.
  • OpenShift-compliant: runs without root, with an arbitrary user ID and on a read-only file system.
  • Backups need just three things: the two databases, the encryption key and the encrypted settings file.

Frequently asked questions

Does BaseLLM run on-prem, on our own servers?

Yes. The BaseLLM AI gateway is installed entirely in your own infrastructure: Docker on a single server, or a Helm chart on Kubernetes and OpenShift. The databases are your own PostgreSQL servers (the Docker package brings its own PostgreSQL).

Does it work with OpenShift security constraints (SCC)?

Yes. It runs without root, with an arbitrary user ID and on a read-only file system. With platform=openshift in Helm, an OpenShift Route is used instead of an Ingress.

How many replicas and how much capacity on Kubernetes?

Start with at least 2 replicas of 1 core + 1 GB each. One replica handles about 400 non-streaming or 100–150 streaming requests per second. Redis is required with more than one replica. Exact capacity is set by a load test in your own environment.

Is there downtime during upgrades?

In a rolling update, the old version does not stop until the new one is ready. On a test cluster, none of the 649 requests sent during an upgrade failed.

What next?

See Install from the Docker registry to pull the package, Get your first trial license for a trial license, and Move from LiteLLM if you are moving from LiteLLM.