LM-Kit One · Coming from OpenAI

From OpenAI, without a rewrite.

Keep the SDK, the prompts and the tooling. Change the base URL, and inference runs on infrastructure you control.

OpenAI SDKs, unmodified Streaming, tools, JSON schema Free to build and evaluate

The whole migration is one parameter.

Every official OpenAI SDK accepts a base URL. Point it at your server, keep the rest of the code.

The official Python SDK, with only the constructor changed.

app.py

from openai import OpenAI

client = OpenAI(
    base_url="http://your-server:5189/v1",
    api_key="your-server-token",
)

reply = client.chat.completions.create(
    model="qwen3.5:9b",
    messages=[{"role": "user", "content": "Summarize this contract clause."}],
)
print(reply.choices[0].message.content)

The official .NET SDK, with the endpoint set in the client options.

Program.cs

var client = new OpenAIClient(
    new ApiKeyCredential("your-server-token"),
    new OpenAIClientOptions { Endpoint = new Uri("http://your-server:5189/v1") });

var chat  = client.GetChatClient("qwen3.5:9b");
var reply = chat.CompleteChat("Summarize this contract clause.");

No SDK at all: the wire shape is the one your tooling already speaks.

terminal

curl http://your-server:5189/v1/chat/completions \
  -H "Authorization: Bearer your-server-token" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3.5:9b","messages":[{"role":"user","content":"Summarize this contract clause."}]}'

Model names come from this server's catalog rather than OpenAI's lineup; list them with GET /v1/models. The coverage matrix states exactly which endpoints and parameters are served.

What survives the move unchanged.

The parts of the OpenAI surface production code actually depends on, tested with the stock clients.

Streaming

Tokens as they decode

Server-sent events with usage reporting, so streaming UIs keep rendering without a change.

Tools

Function calling

Tool definitions, tool choice and parallel calls, exercised end to end by real agentic clients.

Structure

JSON schema, enforced

Structured output is grammar-constrained at decode time, so the completion parses every time instead of usually.

Then the server starts giving back.

An API key swap buys privacy. The application services around the endpoint are what the migration is actually for.

Private

Your boundary, your policy

Loopback by default, tokens hashed at rest, governed egress, and air-gap operation once models are present.

Documents

Answers with page numbers

Upload documents, build searchable collections, and get answers that cite the document, page and passage.

Agents

Defined once, adopted by name

A server-side agent bundles prompt, skill, tools and memory; a request adopts it with one extra field on the same chat API.

Scale

The same stack in production

Start on one machine and scale horizontally as applications, users and document volume grow, without changing the client.

What actually changes.

Three differences worth planning for, before anyone discovers them in production.

Models

Open weights on your hardware

You choose from a curated catalog of open-weight models, or import your own. Quality per task depends on the model and the hardware you give it, so evaluate on your real workload rather than assuming parity with a hosted frontier model.

Coverage

Some endpoints are absent

The dialect covers chat, completions, embeddings, models, files, vector stores and the Responses API, with stated gaps in the matrix.

Capacity

Throughput is yours to size

Concurrency comes from your hardware and the server's slot configuration, not from a metered quota. The console shows the fit before you commit.

LM-Kit One

Change the base URL this afternoon.

Download LM-Kit One See the coverage matrix