lmgw, the local model gateway

lmgw, local model gateway

Your own GPU behind OpenAI- and Anthropic-compatible APIs, with the cloud one alias away.

lmgw is a Linux desktop app. It runs models on your graphics card in Podman containers and serves them on one local port, so tools and SDKs built for OpenAI or Anthropic only need a new base URL.

RPM for Fedora, usually an AppImage too · source on GitHub · MIT

The lmgw dashboard: running models, GPU memory and live requests

One front door for your models

Local models come first. The cloud is there when you want it, behind the same names, and nothing about a client changes when an alias moves from one to the other.

Your GPU, scheduled

Chat, embedding, rerank, speech and image models each run in their own container on llama.cpp, audio.cpp or stable-diffusion.cpp. They start when a request needs them and share the card by measured VRAM.

Two API dialects, one port

OpenAI's chat completions, responses and embeddings next to Anthropic's messages. Any client reaches any chat model, with streaming, tool calls and reasoning translated both ways.

Cloud behind the same alias

Point an alias at an OpenAI-compatible, Anthropic or Gemini model instead, or give a local model a cloud fallback for the hours the GPU is busy with something else.

Voice

/v1/realtime speaks OpenAI's Realtime protocol, over a WebSocket or WebRTC, and answers with your own speech-to-text, chat and text-to-speech models. The Chat has dictation, read-aloud and a voice mode.

Set up by a check

At start lmgw checks the graphics card, driver, container toolkit and disk, says what is missing in plain words and can install the toolkit for you. Find a model turns a name or a Hugging Face link into the file that fits your card.

Tools in one place

Your MCP servers behind one /mcp endpoint, plus built-in tools that search library documentation and your own knowledge bases. Calls can wait for your approval, and long-running ones report back when they are done.

A desktop app

It lives in the tray and opens its own window, which takes the frame, fonts and colours of KDE Plasma. Bound to your network, the same dashboard works in a browser.

Built on an open API

Clients of lmgw get no private channel. The OpenAI and Anthropic dialects, the Realtime WebSocket and MCP are open to any program, and what Kai needed beyond them went in as documented API: the device link with mutual TLS, a Chat feed that also works as a push channel, approvals, late tool results and tools hosted by the device.

The API reference on this site is generated from the gateway's own OpenAPI document, the one a running lmgw serves at /api/openapi.json.

from openai import OpenAI

# Any key works until lmgw requires gateway keys.
client = OpenAI(base_url="http://127.0.0.1:8787/v1", api_key="unused")

reply = client.chat.completions.create(
    model="my-model",  # an alias you created in lmgw
    messages=[{"role": "user", "content": "Hello!"}],
)
print(reply.choices[0].message.content)