Developer hub

Build on AI Server

AI Server speaks the OpenAI API, so the SDKs, frameworks and tools you already use work against models running on your own hardware. Point them at your server's base URL, add an API key, and you're done.

from openai import OpenAI
client = OpenAI(base_url="http://your-server:8080/v1", api_key="<api-key>")
reply = client.chat.completions.create(model="<model-id from /v1/models>",
          messages=[{"role": "user", "content": "Hello from my own hardware"}])
print(reply.choices[0].message.content)

Full reference: API & clients (endpoints, streaming, tool calling, error semantics) · Quick start · All AI Server docs

Open-source samples

Every sample is MIT-licensed — copy what you need into your own product. All of them live in the Software-Tailor GitHub organisation.

ai-server-quickstarts

curl · Python · Node.js · C#/.NET · PowerShell

Minimal, dependency-free examples in your language: list models, chat, stream, embed, and handle rate limits and backpressure correctly.

ai-server-dropin-recipes

Configs · Python

Point tools you already use at your server — Open WebUI, Continue, LangChain, aider, Semantic Kernel — plus a compatibility checker for the official OpenAI SDK.

ai-server-chat-web

HTML · JavaScript · Python

Zero-dependency streaming chat UI — three files, no npm, no framework — whose small proxy keeps your API key out of the browser.

ai-server-doc-qa

Python

Ask questions about your own documents, answered on your server with citations — minimal retrieval-augmented generation in three files, no dependencies.

ai-server-ops-toolkit

PowerShell

One script that answers “why can't the other machine connect?” — listening scope, firewall rules, network profile and authentication, with the exact fix for each failure.

ai-server-agent-frameworks

Python · LangGraph · LlamaIndex · Pydantic AI · OpenAI Agents SDK · smolagents

The same tool-using agent in five frameworks, each verified against AI Server, with the one setting each framework needs.

ai-server-security-agents

Python · C#/.NET

A SOC triage agent on your own hardware: threat-intel, SIEM and ticketing tool seams, sample alerts, model guidance.

ai-server-compat

Python · catalogue

"Works with AI Server": a protocol probe and a licence-checked catalogue of 77 open-source tools, with status per AI Server build.

ai-mcp-registry

Catalogue

The public catalogue of Model Context Protocol tools that AI MCP and MCP-enabled AI Suite apps load — add a tool with a pull request.

Agents and tool calling

/v1/chat/completions supports OpenAI tools, tool_choice and tool_calls, including streamed tool calls, so agent frameworks work unchanged: LangChain and LangGraph, LlamaIndex, Semantic Kernel, Microsoft Agent Framework, CrewAI, Pydantic AI, smolagents and more.

Choose the model deliberately. General instruct models of 7–8B parameters or larger (for example Qwen 2.5 or Llama 3.1 instruct) return reliable structured tool calls. Code-tuned variants often write the call as plain text, and very small models loop or contradict themselves across multi-step tasks.
Cap the loop's steps, and use tool_choice: "required" for a step that must call a tool.
Behind AI Gateway, every response carries X-AISuite-Backend naming the server that answered — useful when debugging a farm.

Works with AI Server

Any tool with an "OpenAI-compatible base URL" setting can use AI Server. The Works with AI Server catalogue lists 77 permissively licensed open-source projects — chat UIs, coding assistants, agent frameworks, RAG, evaluation, security and more — with the setting to use for each and its status against a given AI Server build: verified end to end, configuration documented, or expected to work because every protocol check it depends on passes. Run the probe against your own server to see the same picture for your build.

Settings that trip people up

The base URL ends in /v1 for most tools (Semantic Kernel adds the path itself — give it the host only).
aider needs the openai/ prefix on the model name.
From inside a container, use the server's LAN address or host.docker.internal, not localhost.
Browser-only apps need a same-origin proxy: AI Server does not send CORS headers, which also keeps API keys out of page JavaScript.
On the Free tier, generic clients are rate-limited; any paid tier removes the limit. See Licensing.

Deploy

Windows desktop app from the Microsoft Store; Docker images softwaretailor/aiserver and softwaretailor/aigateway on Docker Hub; a Helm chart (helm repo add softwaretailor https://softwaretailor.com/charts); and bring-your-own-licence offers on Microsoft and AWS Marketplace. Start with the deployment planner and the Kubernetes guide.

Building a product on AI Server?

Technology partners get 12-month development licences, a "Works with AI Server" listing and co-marketing; OEM partners can ship AI Server inside their product.

Partner Programme Apply as a technology partner