AI Server docs / API & clients

API & clients

AI Suite apps need zero configuration — they discover the server themselves. This page is for everything else: your own code, OpenAI-compatible tools, and scripts.

Authentication

Authorization: Bearer <api-key>

API keys are created on the server's API keys page (or its CLI) and sent as a standard bearer token. Loopback calls on the server's own machine need no key. A gateway accepts the same keys as a single server — clients never know the difference.

Endpoints (OpenAI-compatible)

Endpoint	What it does
`POST /v1/chat/completions`	Chat, including streaming (SSE), tool calling, and vision (`image_url` content parts).
`POST /v1/embeddings`	Text embeddings.
`POST /v1/images/generations`	Image generation — text-to-image plus image-to-image / inpainting via base64 init and mask images in the body.
`POST /v1/audio/speech`	Text-to-speech, streamed audio.
`POST /v1/audio/transcriptions`	Speech-to-text (multipart file upload).
`/v1/audio/transcriptions/stream`	Live speech-to-text over WebSocket (binary PCM frames up, transcript deltas down).
`GET /v1/models`	Models available to serve; a gateway answers with the union across its worker farm.

Additional native endpoints (voice cloning, vision analysis, model install/pull, usage, metrics) exist alongside these; the in-app Help lists the full surface, and a machine-readable spec is planned.

Point standard tooling at it

# Python (openai SDK)
from openai import OpenAI
client = OpenAI(base_url="http://your-server:8080/v1", api_key="<api-key>")
r = client.chat.completions.create(model="<model-id>",
        messages=[{"role": "user", "content": "Hello"}], stream=True)
for chunk in r:
    print(chunk.choices[0].delta.content or "", end="")

# curl
curl -N http://your-server:8080/v1/chat/completions \
  -H "Authorization: Bearer <api-key>" -H "Content-Type: application/json" \
  -d '{"model":"<model-id>","messages":[{"role":"user","content":"Hello"}],"stream":true}'

Tools with an "OpenAI-compatible / custom base URL" setting (chat UIs, coding assistants, editor plugins) work the same way: base URL http://your-server:8080/v1, your API key, pick a model from /v1/models.

Error & streaming semantics worth coding for

429 — you hit a rate limit, quota, or the free-tier meter (below). When upgrading would lift the limit, the response carries X-AISuite-Upgrade: 1.
503 + Retry-After — honest backpressure: every worker is saturated. Wait the indicated seconds and retry; do not hammer.
Streaming is standard SSE (stream: true). Keep proxy buffering off anywhere you put your own proxy in front.
Behind a gateway, transient worker failures are retried server-side before you see them; responses carry X-AISuite-Backend naming the machine that answered — useful in bug reports.

Free-tier metering for generic clients

On the Free tier, AI Suite apps are unlimited, but generic clients (curl, SDKs, third-party tools) are metered to a small allowance — currently 1 request/minute and 10/day — after which the server answers 429 with X-AISuite-Upgrade: 1. Any paid tier removes the meter entirely. The line exists so the free tier can power the app suite without giving away an unlimited general-purpose inference server.

Sample apps (open source)

Working code you can run against your own server in a few minutes, published under the MIT licence — copy any of it into your own project.

ai-server-quickstarts — the endpoints above in curl, Python, Node.js, C#/.NET and PowerShell. Each file is dependency-free and shows model discovery, streaming, and the 401 / 429 / 503 handling described on this page.
ai-server-chat-web — a streaming chat UI in three files, with no npm or framework. It includes a small loopback host, because browser JavaScript cannot call the server directly (see below) — which also keeps your API key out of the browser.
ai-server-doc-qa — ask questions about your own documents, answered locally with citations, using /v1/embeddings together with chat. Minimal retrieval-augmented generation, no vector database.
ai-server-dropin-recipes — point tools you already use at your own server: Open WebUI, Continue, LangChain, aider, Semantic Kernel. Includes a compatibility checker for the official OpenAI SDK.
ai-server-ops-toolkit — a script that diagnoses why a server can't be reached (listening scope, firewall rules, network type, authentication) and prints the fix for each problem it finds.

Calling from a browser

AI Server does not send CORS headers, and its OPTIONS preflight requires authentication — which browsers never send on a preflight. Page JavaScript served from another origin therefore cannot call the API directly. Call it from your backend instead, or put a small same-origin proxy in front of it (the chat sample above ships one in about 150 lines). Keeping the key server-side is the better practice regardless: a key placed in page JavaScript is visible in developer tools, in localStorage, and in any screen-share.

Legacy compatibility endpoint

Servers upgraded from AI Server 2.0.1 keep serving that generation's local-engine wire format so old setups don't break. This compatibility surface is sunset on 2026-12-31 — move generic integrations to the /v1 endpoints above before then.

Next: Gateway & operations · back to docs home