AI Server docs / Quick start

Quick start

Three routes to a working server, fastest first. All three serve the same APIs to the same clients.

1. Desktop (Windows)

1. Install AI Server from the Microsoft Store.
2. Launch it. The app starts and manages its local daemon; the Server page shows status, bind address, and connected clients.
3. That's it for local use: every AI Suite app on the same machine discovers the server automatically and offloads its AI work to it. The free tier covers this fully.
4. To serve other devices on your network (a paid feature): Server page → Server settings, choose the LAN or all-interfaces bind, and create at least one API key on the API keys page. The server refuses a network bind without a key — that is deliberate.

2. Single Docker container

A container binds the network by definition, so it needs a paid license key and an API key (see Licensing & activation). The image runs as a non-root user and keeps all state in the /data volume.

docker run -d --name aiserver \
  -p 8080:8080 \
  -v aisuite-data:/data \
  -e AISUITE_ENGINE=real \
  -e AISUITE_LICENSE_KEY_FILE=/run/secrets/aisuite_license \
  softwaretailor/aiserver:latest

Notes that save time:

• AISUITE_ENGINE=real serves real local models — the first chat pulls a multi-gigabyte model into the volume, so opt in deliberately. The default engine is a stub for wiring tests.
• Prefer AISUITE_LICENSE_KEY_FILE (a mounted secret) over AISUITE_LICENSE_KEY — an env var is visible in docker inspect.
• Probes: GET /livez (liveness — 200 while the process runs, even when draining) and GET /readyz (readiness — 503 once draining). Route on /readyz, restart on /livez, never the other way round.
• Logs are JSON per line on stdout (AISUITE_LOG_JSON=1 is the image default) — ready for Fluent Bit / Loki / CloudWatch.
• Verify: curl -fsS http://localhost:8080/readyz, then open http://localhost:8080/dashboard in a browser.

3. Compose farm with AI Gateway

A gateway container (AISUITE_ENGINE=gateway) in front of two worker containers gives you load balancing, health-based routing and zero-downtime upgrades on one host. Clients connect to the gateway exactly as if it were a single AI Server.

services:
  gateway:
    image: softwaretailor/aigateway:latest
    ports: ["8080:8080"]
    volumes: ["gateway-data:/data"]          # gateway.json (the worker pool) lives here
    environment:
      AISUITE_ENGINE: gateway
      AISUITE_LICENSE_KEY_FILE: /run/secrets/aisuite_license
    secrets: [aisuite_license]
    stop_grace_period: 40s
  worker-1:
    image: softwaretailor/aiserver:latest
    volumes: ["w1-data:/data"]
    environment:
      AISUITE_ENGINE: real
      AISUITE_LICENSE_KEY_FILE: /run/secrets/aisuite_license
    secrets: [aisuite_license]
    stop_grace_period: 40s
  worker-2:
    image: softwaretailor/aiserver:latest
    volumes: ["w2-data:/data"]
    environment:
      AISUITE_ENGINE: real
      AISUITE_LICENSE_KEY_FILE: /run/secrets/aisuite_license
    secrets: [aisuite_license]
    stop_grace_period: 40s
secrets:
  aisuite_license: { file: ./license.key }
volumes: { gateway-data: {}, w1-data: {}, w2-data: {} }

The gateway reads its worker pool from gateway.json in its data volume (workers, bearer tokens, routing strategy). One license key covers the whole fleet — each node activates itself independently. The full annotated version of this scenario, including the gateway.json, ships with the product's deployment recipe library.

Check the farm: http://localhost:8080/dashboard shows both workers, their health, in-flight load and model catalogs; every proxied response carries an X-AISuite-Backend header naming the worker that served it.

Where next

• Production cluster → Kubernetes & Helm
• Keys, tiers, offline behaviour → Licensing & activation
• Routing strategies, canary, monitoring → Gateway & operations
• Point your own code or tools at it → API & clients