lmgw and Kai

Releases

What changed in each release of lmgw and Kai. Every release on GitHub carries an RPM (most lmgw releases also an AppImage), and an installed RPM offers the next release by itself.

Jump to lmgw · Kai. The releases of Kai mobile are on GitHub, those of Kai for Firefox too.

lmgw

lmgw 0.7.0

Latest

lmgw on Windows, as a native app with its own installer and updater; lmgw as a Podman image for a server; and a VRAM admission that starts what would run, with a warning, instead of refusing it.

  • lmgw on Windows. A per-user installer (no administrator needed) and an updater that checks the download's sha256 before it starts the installer. There is no Podman on Windows: llama.cpp, stable-diffusion.cpp and audio.cpp run as native processes, each in a Windows job that ends with lmgw, from the engines' own release builds.
    • Engines download for you. lmgw picks the build per engine from the card and driver (CUDA 13, CUDA 12 where it reaches the card, Vulkan, CPU), checks it against its pin and unpacks it under %LOCALAPPDATA%\lmgw\engines. A CUDA build with no kernels for the card is noticed, and the start goes on with the Vulkan build. The Engines page has a Windows tab with each build, why it was chosen and Try CUDA.
    • It looks like Windows. The window follows the system's dark or light mode and the accent colour, and the title bar takes the sidebar's colour.
    • The machine check, VRAM accounting (per-process figures from the driver), the private files and keys (owner-only ACLs, Credential Manager) and links (the shell's open verb) all have Windows versions.
    • Not yet on Windows: agents, git engine builds, PDF text extraction, the tray menu in dark mode and the Windows 10 theme path.
  • lmgw as a server image. lmgw-server is the gateway and dashboard without the tray app, in a Podman image that runs its model, agent and build containers with a nested Podman, started by a rootless Quadlet unit. A start with a pending migration first copies the database to backups/. Settings the unit pins show read-only, with Settings → About saying where each came from. docs/server.md is the guide.
    • TLS on the main port (TLS 1.2 and 1.3, the pair reloaded when its files change) and proxy mode for a trusted reverse proxy's forwarded scheme, host and client.
    • Accounts. Names and passwords (argon2id) with sessions, set-password links, a sign-in throttle and lmgw-server account commands. The server always has them; the desktop has a switch under Settings → Network & access → Accounts. Keys and devices say who made them.
    • Audit log. With accounts on, every administrative write, writing tool call and sign-in is an entry, secrets redacted, kept for a number of days you set, shown as the Activity list.
    • The device CA's key is sealed under the image's wrapping key, and the update notice and the machine check speak for the server.
  • VRAM admission starts what would run. A model that does not wholly fit the card's free memory is no longer refused when it can work partly on the CPU; it starts and says "It will work but only fits partially on the GPU, expect slowdowns", on the Models page, in the start answer and in the test.
    • Image rows with offload_to_cpu or CPU/disk parameters are charged their on-card part, learned per engine build. A pipeline that is all on the card and larger than the card is still refused, with what offloading would do.
    • llama.cpp rows are charged what lands on the card: a partial --n-gpu-layers, --no-kv-offload and MoE expert offload. A row that llama.cpp's --fit would shrink is fitted by lmgw (context, then experts, then layers) against its own figures, and the start renders the result as explicit flags.
    • Several cards in one engine's view are not fitted yet.
  • Auto headroom. Under Settings → GPU → Admission the headroom is auto by default: the largest rise of other programs' memory use over the last 7 days plus a margin per engine class while a model starts, or the fixed value. It uses the fixed value until a day of readings exists. Forget learned growth… clears the history.
  • MoE offload. Chat rows have Experts kept in memory (--n-cpu-moe) and GPU cache for experts in memory (--moe-cache-mib), and admission charges only what stays on the card, by tensor name, draft model included.
  • Dashboard. Settings → About; read-only pinned settings; the Activity list; the Accounts card.

Breaking / upgrade notes

  • Clients: Kai, Kai mobile and Kai for Firefox move to this release. The update manifest gains an nsis entry beside rpm, and the lmgw-update crate's artifact type changed with it.
  • Migrations 0101 to 0106 run on start: accounts and sessions (0101), the audit log (0102), image idle and working-set learning (0103, 0104, 0105: an image row's learned peak and budget now belong to the engine build they were learned under and are learned again after an engine change) and the outside-use history (0106).
  • New settings and fields: vram.headroom_mode (auto or fixed) beside headroom_mb; n_cpu_moe and moe_cache_mib on chat rows; trusted_proxies, tls.* and accounts.audit_days. Warnings on start/restart answers and image tests; the op list has engine_releases.
  • Auto headroom replaces the fixed one by default; set headroom_mode to fixed to keep the value you had.
  • The server image pins its listeners and keys; on the desktop an account switch starts at off.

Fixes

  • A partial image start is held to what admission gave it and learns from a failed generation; a fitted llama-server container is adopted at boot.
  • A first llama.cpp start under a declared budget is no longer refused as too large before the cards are known.
  • A new row's id is settled under the routing-write lock, so two Get it runs cannot pick the same one.
  • A size over max_body_mb answers 413 on Windows, not a reset; the main TLS paths must be absolute; a download without a dialog goes to the Downloads folder.
  • Podman's known-harmless stderr lines are no longer quoted back as errors.

lmgw 0.6.0

A machine check that sets up the graphics card, Find a model, the OpenAI platform as its own upstream, Anthropic's blocks carried natively, image and audio tools in the Chat, and the Chat as a crate other apps can host.

  • The machine check. At start lmgw checks whether this computer can run local models and says what is missing in plain words: the graphics card and its driver (Secure Boot included, amdgpu on AMD), NVIDIA's container toolkit, Podman, the model settings against the card, free space and the engine downloads. It shows as the "This computer" card first in Settings → GPU, as a card on the Overview and above the Chat's empty state, and as a banner in the Chat while local models are involved (a cloud-only setup is not nagged; "Not now" quiets all three). The tray adds Graphics card needs attention… when the verdict leaves ready.
    • Set it up installs the toolkit through a fixed root helper, /usr/libexec/lmgw/lmgw-machine-helper, behind lmgw's own confirmation (every command and file under "Show the commands") and one polkit password prompt. It runs only from the app's window and as a system task (lmgw-machine-setup.service), so closing lmgw doesn't interrupt it. Where the helper can't apply (no dnf, an image-based system) the check lists the commands to type instead.
    • lmgw writes the card's CDI description itself, without root, into its own runtime directory, rewrites it after a driver or package update and passes it to its containers with --cdi-spec-dir; there is no nvidia-ctk cdi generate to run for them.
    • Model settings made for the other vendor's card (AMD Vulkan or NVIDIA CUDA) can be switched in one confirmed step, with every change listed.
    • A start the computer can't make answers 503 gpu_setup with the reason, or the model's fallback answers it. The check and its fix are also the machine_check op and the lmgw__machine_check tool.
  • Engine images arrive with the model. A model download also fetches the image its class runs in, and a start whose image is missing downloads it before admission, showing its bytes, without taking a place in the queue. A failed image download answers engine_download.
  • Find a model. Models → Find a model takes a name as someone said it ("qwen 3.6") or any Hugging Face link, searches the hub live (originals before uploads, no model list of lmgw's own) and answers one card: the original, the upload and the file that fits this machine's card, judged by the listed sizes and then a Range read of the GGUF header, before anything downloads. Get it downloads the file with its companions, plans it, fits its context to the card and creates the row without starting it; Say hello opens a thread. Send to a friend copies the model's name and its public Hugging Face address, and a pasted share text plans that exact file. Add from Hugging Face marks every file by its fit and preselects the recommended one. Self-admin agents get lmgw__model_find, lmgw__model_get and lmgw__model_get_status.
    • Downloads resume from their .part after a restart, are checked by length and sha256 before they are renamed into place, and pause on a metered connection.
    • A fresh install needs no settings to download: it keeps models under ~/.local/share/lmgw/models/, one folder per class, shown and changed under Settings → Runtimes.
  • Binary units everywhere. Every byte figure lmgw shows or says (the machine check, downloads, image pulls, disk, card memory, body sizes, notes, tool sentences) is in KiB, MiB, GiB or TiB, through one formatter.
  • The OpenAI platform gets its own egress. The openai protocol now means api.openai.com only; every other OpenAI-shaped provider is openai_compatible.
    • /v1/responses goes to the platform natively, metered and priced, unless only lmgw's loop can serve the request (an MCP label lmgw serves, a previous_response_id lmgw stored). Hosted tools pass through on the native path. Stored responses belong to the key that made them, and GET/DELETE/input_items on a native id go through its upstream.
    • Chat sends to the platform go over the Responses API unless they set stop, seed, a penalty or carry audio. Encrypted reasoning survives across turns: in the Chat, as a thinking block's signature on /v1/messages and as encrypted_content on /v1/responses. A model's refusal is its own part.
    • Service tiers. lmgw asks for service_tier: auto when the client set none and prices each answer at the tier it reports. Price rows have a tier; traffic at a tier with no price stays unpriced with a note, and the Prices page lists it.
    • Budget floors. A tier with no price, or a stream cut before its usage, still counts against budgets at a floor (the standard price, or a local token count). A budgeted caller is refused with budget_unpriced on a model with no price for what it is billed in, flex without a flex price included.
    • max_completion_tokens for the output cap; llama.cpp samplers are left out and named in x-lmgw-params-ignored; OpenRouter's reasoning object becomes reasoning_effort; OpenAI's error codes, Retry-After and x-ratelimit-* reach the client.
  • One fallback per model and alias. Every model and alias has one Fallback setting, used for every reason it can't answer: the GPU hold, a benchmark, VRAM taken outside lmgw, a deferred background request, a card that isn't set up, an exhausted quota, an unreachable upstream or a disabled row. A fallback is served as if the client had named it, so it can hand on to its own; x-lmgw-fallback-chain lists each hop with its reason. A fallback must be of the same model type and may be a local model; a loop is refused. lmgw remembers an upstream that is out of quota and skips straight to its fallback until a re-check (quota_recheck_seconds, under Usage & cost) or Check now.
  • Anthropic, natively. Between a Messages client and an Anthropic upstream, lmgw now carries what it used to flatten, whole and streamed: thinking blocks with their signatures, blocks it doesn't model (redacted_thinking, server tool blocks, newer types) verbatim, text block citations, compaction blocks and context_management edits, and Files API sources for images and documents. Every other route gets the nearest equivalent (the compaction summary as text, citations dropped) and x-lmgw-params-ignored names what was left out. Signed and verbatim blocks go back only to the upstream that produced them. The Anthropic dialect of /v1/models carries Anthropic's capability blocks.
  • Stalled streams end. An upstream that stays silent past its timeout ends the client's stream with the protocol's own error. Keep-alive comments only hold a stream open while the model may be thinking out of sight or a provider-run tool is running.
  • Structured output, translated everywhere. response_format, text.format and output_config.format are one field that every egress writes in its upstream's form (Anthropic output_config.format, Gemini responseJsonSchema). A model whose capabilities say it can't is refused early with response_format_unsupported; upstream schema refusals arrive as invalid_json_schema or response_format_unsupported with param.
  • Image and audio tools for the Chat. image__generate, image__edit, image__upscale, audio__speak and audio__transcribe are a built-in toolset on /mcp and, in Chat threads, the labels image and audio. A running call shows its stage, step progress and previews and can be cancelled. A generated picture gets a card with Re-roll, Vary, Edit and Upscale that work without the chat model (POST /chat/api/attachments/{id}/media). Default models per tool are set in Settings → Chat, a folder or a thread.
  • The Chat on the Chat plane. The dashboard's Chat reads only /chat/api/* and /v1 routes, scoped to the caller (new reads for its settings, toolsets, tools and knowledge bases), and is a crate of its own, lmgw-chat-ui, with a narrow layout down to 280 px and no inline scripts, so a paired device can host the same Chat. Kai for Firefox shows it in a Firefox sidebar.
    • A message can carry its origin, where the user writes from, kept out of the system prompt.
    • Conversations can be renamed (menu or F2; POST /chat/api/threads/{id}/title), New chat starts in the open thread's folder, the composer is one input bubble, and side panels can be dragged wider or narrower.
  • Device-to-device transfers. A paired device can offer a file to another device of the same app (an app id declared at pairing), relayed through lmgw: a resumable ranged upload checked by sha256 (/transfers/api/transfers), the offer on the Chat feed as transfer.updated, a ranged download, and expiry. The Devices card gets a Transfers section, and enrolment carries the device's own name.
  • API compatibility checks. Every gate run checks each body lmgw sends upstream and each /v1 answer and stream event against OpenAI's OpenAPI spec, Anthropic's SDK schemas and Gemini's discovery document. scripts/compat-live.sh drives multi-turn scenarios through the official SDKs against a running lmgw; its results are published at https://lmgw-dev.github.io/docs/compat/. New on the way: POST /v1/responses/input_tokens, and a sampling parameter a model refuses is retried once without it, left out from then on and named in x-lmgw-params-refused.
  • Background requests. x-lmgw-background: on makes one request a guest on the GPU: it uses a local model only where that disturbs nobody, otherwise the fallback answers or a 503 gpu_deferred says to retry. The Models page shows a model's deferrals over the last 24 hours.
  • The Voices lab. Under Labs: rounds of candidate voice samples from any text-to-speech alias, saved to the voice library with their exact transcript (also lmgw__voice_generate, lmgw__voice_sample_save).
  • Audio. /v1/audio/alignments answers one typed shape whatever model aligned (seconds, the sample rate, each word's character span in the submitted transcript), and every local audio row publishes its languages.
  • Dashboard. Engine builds and images are Engines (Overview's section is Running models); the sidebar nests Labs and Downloads; messages name settings by their labels (Settings → GPU → Admission → Load timeout), not config keys; the window remembers its size.

Breaking / upgrade notes

  • Clients: Kai 0.3.0, Kai mobile 0.2.0 and Kai for Firefox 0.1.0 are the releases built against this one; older Kai builds pin lmgw 0.5.0.
  • openai rows become openai_compatible (migration 0090), whatever their host, and the start notice names each one. Switch an api.openai.com row back to openai on the Upstreams page to get the platform egress. supports_responses is an openai_compatible choice only and is cleared on Anthropic and Gemini rows; audio_cpp rows must be openai_compatible.
  • The global hold fallback and inherit are gone (migration 0095). Each row keeps the fallback it answered with, the start notice names the rows whose fallback came from the global, and a fallback of another model type is refused at the row's next save.
  • Budgets count only what is priced in full: a budgeted caller is refused on models with no price for what they bill (a price of 0 is allowed).
  • Answers changed: an upstream 503 is 502 api_error (only a 529 is overloaded_error); rate and quota refusals carry OpenAI's own code and type, an exhausted quota on /v1/messages is 402 billing_error, and an OpenAI 4xx lmgw has no word for is relayed as sent; languages moved from capabilities.speech to the top of capabilities; /v1/audio/alignments is lmgw's typed shape, no longer audio.cpp's JSON. New codes: gpu_setup, engine_download, budget_unpriced.
  • A silent upstream now ends a stream after its timeout, where keep-alive comments used to hold it open.
  • Migrations 0085 to 0100 run on start. The RPM depends on polkit and ships the root helper and its polkit action (io.github.lmgw-dev.lmgw.machine-setup); building the RPM by hand needs cargo build --release -p lmgw-machine-helper before cargo tauri build, as ci/build.sh does.

Fixes

  • Every answer from the gateway's origin except the dashboard's own sends nosniff and Content-Security-Policy: sandbox, so stored or relayed content can't run as the dashboard.
  • The updater runs pkexec and dnf by absolute path and downloads the RPM into a private directory instead of a fixed /tmp path.
  • The RPM no longer ships group- or world-writable files.
  • A streamed chat stops its model as soon as the client goes, even while the model is prefilling.
  • A split GGUF is charged for every part, not only the first.
  • A tool call with no arguments goes upstream as {}; an MP3's input_audio goes as mp3; the /v1 answers carry the fields OpenAI's and Anthropic's schemas require (logprobs: null and others).
  • Chat attachments, lab files and voice clips answer Range requests with 206.

lmgw 0.5.0

Realtime voice over WebRTC, an encrypted and mutually authenticated link for paired devices, decision models, and labs that keep their history. lmgw is MIT-licensed from this release.

  • WebRTC realtime calls. POST /v1/realtime/calls is OpenAI's WebRTC transport for /v1/realtime sessions, in OpenAI's shape: post an SDP offer (raw application/sdp, or multipart with a session part) with your key and get the answer back. Audio is Opus over UDP, encrypted with DTLS-SRTP; events travel on the oai-events data channel; POST /v1/realtime/calls/{call_id}/hangup ends a call. OpenAI's browser example and the Agents SDK's OpenAIRealtimeWebRTC work with the base URL pointed at lmgw. lmgw answers as ICE-lite with one host candidate, the address the POST arrived on, and has no STUN or TURN: the client must reach that address directly, on the LAN or through a VPN. The UDP port is the HTTP port's number unless Settings → Realtime → WebRTC calls names another; that group also lists the live calls.
  • The device link. Paired devices connect to a TLS listener of their own and authenticate with a certificate; bearer device keys are gone. The plain listener stays HTTP for local OpenAI clients and the dashboard.
    • lmgw runs a small private CA. The pairing QR code or link carries the CA's fingerprint, a one-time token and the URLs to try (LAN, VPN). The device makes its own key, enrols the public half with the token and gets a certificate, and both sides check each other from then on.
    • Each enrolment is confirmed on the Devices card: the device and the card show the same short code, and the device opens nothing until you click Confirm. The card also copies the CA certificate and rotates the CA. The CA's key is encrypted under the desktop's Secret Service.
    • The listener is off until you pair a device; the pairing form offers to turn it on and, for another machine, to widen it. Its port is the HTTP port's number plus one (Settings → Network & access → Devices).
  • The Chat feed is a push channel for devices. GET /chat/api/feed takes keepalive_s (up to chat_feed_keepalive_max_s, default 900), so a phone can hold the connection open in the background. hello carries the keep-alive, the device's URLs, certificate and open_approvals; a live caught_up ends the catch-up; every message write is recorded as message.added, message.updated or message.deleted.
  • Device-hosted tools and the device-tools grant. A key's policy gains device_tools, the device labels (for example ["kai"]) whose hosted tools it reaches; the tool scope then filters them like any server's. An allow pattern such as kai__* no longer reaches a device by itself (the upgrade grants the labels existing patterns named). A Chat thread's attached label that offers nothing says why in the error frame (tool_unreachable, tool_offline, ...).
  • Decision models. POST /v1/systemone relays the System One wire (a state and typed questions, answered with calibrated probabilities) to a local llama.cpp decision model (b11361 or later) or to an OpenAI-compatible upstream such as OpenRouter. A GGUF whose header declares decision.type is a third aux kind, published as decide/<id>; aux rows gain ubatch_size, batch_size, mmproj_path, backend and threads. OpenRouter's decision, embedding, rerank, speech and image models now appear in /v1/models under their own tasks.
  • The lab store and the Decision lab. The Decision, Image and Audio labs keep their runs in lmgw's database. The new Decision lab asks a decision model typed questions, compares and perturbs inputs, and saves question sets with a confidence threshold per question. A set run sends a CSV or JSONL file through a set as a job and reports accuracy, a confusion matrix, reliability bins and the threshold curve; batch agents can classify through a set. The Image and Audio labs save the files their runs made (one file, or a zip with a manifest).
  • Save As dialogs. Every download in the desktop app asks where to save, through the desktop portal where there is one.
  • VRAM and plan accuracy. A chat or aux row's footprint charges llama.cpp's compute buffers, and the KV cache follows llama.cpp's layout (sliding windows, shared layers, no V cache under MLA, MTP and draft contexts). A plan that cannot fit the card says so, and plans load nothing.
  • GGUF licence badges. The Models page shows each file's licence from its header beside the row.
  • MIT licence. lmgw was FSL-1.1-MIT; it is plain MIT now.

Breaking / upgrade notes

  • Paired devices must pair again. Every existing device row becomes v1: its bearer opens nothing, and the refusal says so. Pair again from the Devices card and confirm the code; v1 pairing links are refused. key_rotate refuses a device.
  • Kai needs 0.2.0, which pairs over the device link. Kai mobile 0.1.0 is released alongside.
  • VRAM estimates rise for chat and aux rows (compute buffers), so admission can refuse or evict where a start used to fit.
  • Removed spellings are refused, naming the valid one: the op embed_model_set (use aux_model_set); embed as an aux target (use aux); upstream_set protocol openai with kind llama_server (use llama_cpp); the probe parameter of the model plan tools.
  • Run arguments are lists of strings (extra_run_args, extra_args) on every tool, op and answer; the newline or shell text form is refused.
  • Answers changed: key_create answers key (was plaintext); routable is routable_candidates; unknown values are left out instead of "" or null; unknown ids are 404 not_found on the docs routes; a failed dictation answers {code, message}; the self-admin tools use the dashboard's names and types.
  • Migrations 0079 to 0084 run on start. Building needs opus-devel; the RPM depends on opus.

Fixes

  • Container starts no longer pass the desktop app's open files or LD_LIBRARY_PATH to pasta, which caused SELinux denials.
  • A failed agent run no longer keeps the agent's token in its error text.
  • Saving an alias no longer resets its min-p, repeat, presence and frequency penalties.
  • Price sync leaves models billed per character, second or image unpriced instead of pricing them wrong.
  • The Audio lab's voice list starts no model when the page opens.
  • A voice-mode reply bubble stops spinning when its done frame is bad.

lmgw 0.4.4

lmgw's own HTTP API is documented and typed, and a developer document covers what clients, devices and agents can call.

  • /v1/openapi.json is the developer document. It lists every operation a non-owner credential reaches: the inference routes, the Chat API that paired devices use, the agent run API, and GET /api/version, with a map of which key kind may call what. The owner's admin plane stays in /api/openapi.json.

  • Documented for the first time:

    • the rest of the Chat API: thread create, read, settings, delete, pin, move, archive and keep; send, continue, regenerate and edit with their SSE frames; message delete, read-aloud, voice warm-up, speech stop, search, folder patch and delete, attachments, and the exports (including the lmgw.chat.v1 bundle);
    • the Knowledge bases API, the Audio lab and Image lab APIs, and GET /chat/api/threads/rows.

    Only the dashboard's pages, the /ui redirects and the agent app and MCP proxies stay undocumented.

  • Typed answers. Every answer, op result and SSE frame of lmgw's own API is an lmgw-api-types type that the handler builds, and the schema comes from that type. Fields the server always sends are listed as required; request bodies of shared types are <Name>Input components. audio.cpp's own JSON (/v1/tasks/*, /v1/audio/alignments) is relayed verbatim and documented as such. The /v1 protocol routes still pass unknown request parameters to the upstream.

  • Kept honest by tests. Every drift test validates the live answer against the built document, including every Chat turn frame and the /v1/chat/completions stream. A test refuses new untyped answers, and a check holds the count of inline JSON in handlers and ops to a ratchet (README, "Typed answers").

  • Behaviour changes, malformed input and error paths only:

    • A JSON body of the wrong shape is 422 bad_request on every Chat route (send's voice, continue, regenerate and the job answer were 400). Invalid JSON stays 400.
    • Audio lab bodies of the wrong shape are 422 {error}. Before, a non-string transcript cleared the clip's transcript and a non-string alias fell back to another model.
    • GET /audio-lab/api/refs/{name} errors are JSON {error} (were plain text). The Audio lab's speech and task hand-offs answer a bad body in the OpenAI envelope. Image lab errors gain param, type and code where they lacked them.
  • Fixes: a tool turn could overflow a 2 MiB stack in a debug build; GitHub CI runs the full test suite again.

lmgw 0.4.3

Personality profiles, approvals and late results for MCP tools, and client devices that host tools of their own.

  • Personality profiles. A profile bundles a system prompt, reasoning settings and a voice, and shapes a thread's turns.
    • Assign one to a thread or a folder's defaults; the chat_profile setting names the Chat's profile for new threads. A folder with no profile inherits the Chat's; it does not mean "none". A spoken session follows its thread's profile from the next response.
    • Edit them at /chat/profiles (also Settings → Chat profile), with a drawer and folder picker, header chip and voice-panel chip in the Chat. Preview, voice Test and Speak work on an unsaved draft; a built-in profile can be reset. Unsaved edits ask before another profile opens, and Discard on a new profile drops the draft.
    • The self-admin tools lmgw__profiles, lmgw__profile_set and lmgw__profile_delete manage them. A device may change a profile that steers the admin tools only while it may write with them (403 profile_in_admin_use), and a device's TTS alias is checked against its key's scope.
  • Approvals for MCP tools. require_approval on a thread (or folder default) stops a turn at a gated call.
    • The thread reports pending_approvals; POST /chat/api/threads/{id}/approvals decides, and the resumed reply streams on as its starter. A new message declines what waits. approval.requested and approval.decided events on the feed, the tool frame and the realtime mcp_approval_* items carry the call's call_id.
    • The dashboard shows Approve and Decline on the call's card, per card or as a batch.
    • A device can only tighten the rules: a write that gates fewer calls is 403 approval_loosen_refused, and it decides only within its own reach (approval_out_of_scope). The owner's approval floor per thread and folder always holds, under either spelling of a tool's name.
  • Late tool results (MCP Tasks). An MCP server that answers a tool call with a task no longer holds the turn.
    • The result enters the thread when it arrives, as an lmgw__job_result pair placed in chronological order, never while a turn runs, and a send, edit or regenerate keeps it.
    • GET /chat/api/threads/{id}/tasks lists a thread's jobs, POST /chat/api/threads/{id}/tasks/{task}/cancel cancels one, and POST /chat/api/threads/{id}/answer lets the model answer results that arrived while nobody was talking.
    • Bound voice sessions hear lmgw.task.done and the continuation is spoken. The dashboard shows a card (Job · tool · status) and a strip of the thread's jobs with Cancel; the server list counts open tasks (open_tasks on GET /api/mcp-servers).
    • device.revoked is a stored feed event for a deleted paired device, so a host can cancel the jobs that device started. A disable says nothing.
  • Devices host MCP tools. A paired device can serve tools over GET /mcp/host; initialize tells it the link's limits (lmgw/host_limits in _meta). Closes say why (a deleted key is 4003; of two links the newer stays). A run calls only the tools it was offered. The dashboard's reach list shows a device chip.
  • /mcp passes resources through, namespaced by tool prefix, with MCP Apps metadata: tools/list stamps each tool with _meta["lmgw/server"], and the Chat tool frame gains server_label, ui_resource and structured_content. App-only tools stay out of model runs.
  • Colliding tool names no longer shadow each other. Both tools move to <server>__<tool> (the server's own name still goes upstream; the frame says tool_moved), stable across restarts. A key's deny list holds under every name a tool has; an allow list admits only the name it names; a built-in's switch does not reach a server's tool.
  • structuredContent stays out of the model when a result also has text content: clients and app views get it, the model reads the text.
  • Gemini tool loops keep their thought signatures. The signature rides in the call's id and goes back to Gemini; an unsigned step gets the skip value. Chat completions, Messages and /v1/responses all pass it.
  • Fixes.
    • The Chat page follows results stored before its own send in place, without a reload.
    • Test flakes were fixed (a Stop sent before a start, the VRAM sampler, realtime and agent waits); scripts/flake-hunt.sh runs the suite under load.
    • The self-update code moved into the lmgw-update crate; the feed's auth header goes only to the feed's own origin.

Upgrading: migrations 0071 to 0078 run on start (profiles, device MCP rows, tasks, pending approvals, a message's task, the approval floor, a message role index, known tools). Tools whose names collided are now prefixed by their server, so clients that called the bare name of the shadowed tool should list tools again.

lmgw 0.4.2

Client apps as paired devices, prices beyond tokens, and a Chat that follows every client.

  • Client apps as paired devices. Pair a client (a desktop voice client first, a phone later) from Usage → Keys → Devices with an lmgw-pair: link.
    • Each device gets its own key, tool scope and budget. It reaches the Chat API and /v1, and nothing of the admin plane.
    • Disable, Rotate and Delete end its connections at once. The realtime close says why: 4003 with a kind token for a revocation, 4004 for a thread out of reach.
    • A device can be given its own level of lmgw's admin tools (off, read only or full), capped by the gateway's own setting.
  • A Chat change feed and ongoing folders. GET /chat/api/feed streams thread and folder changes, running turns, voice sessions and the GPU hold, and resumes across restarts.
    • A folder can be one ongoing conversation that every client continues (POST /chat/api/folders/{id}/current). A new thread starts after the folder's idle time or on "New conversation".
    • The new lmgw-client crate holds the realtime protocol, the voice state and the feed reader for clients; the dashboard's voice panel runs on it.
  • The dashboard's Chat follows other clients live. A chat frame on /api/events names what changed, and the page reads it again without leaving the tab. A draft whose conversation was deleted elsewhere waits, unsaved, until you send it.
  • Prices beyond tokens. A model can be priced per audio minute, per 1M characters, per generated image and per request, beside its token rates.
    • Requests record the audio length, characters and images they measured. Nothing is estimated: a quantity lmgw can't measure stays unknown, never 0.
    • Usage and Traffic show the quantities.
    • The catalog sync brings in request fees and cache prices, and a fallback is priced as the alias that answered.
  • Supertonic speaks typographic text. A reply containing a character the model's vocabulary lacks, such as „ or an emoji, no longer fails its speech. Such characters become plain equivalents or are dropped, and x-lmgw-speech names them. A text with nothing left to say is empty_input on every TTS model.
  • No more "database is locked". Every write takes SQLite's write lock up front, so concurrent writes wait their turn instead of failing.
  • MCP servers connect or fail. An HTTP/SSE server's handshake is bounded by its timeout_ms.
    • Podman-isolated servers run in labelled containers, which are removed when the server stops and swept at the next start.
    • Settings saves and the GPU hold no longer wait for MCP reconnects.
  • Streams follow stream_options.include_usage. Streamed chat completions end with a usage chunk only when the client asks for it, as OpenAI's do.
  • A clean shutdown. Quit, an update's restart and SIGTERM stop the gateway first: feeds and voice sessions close with 1001, and cut turns keep their partial reply.
  • The API reference reads as documentation for client developers.

Upgrading: migrations 0061 to 0070 run on start. If an old price row used a removed unit (per_second, per_char), lmgw refuses to start and names the row. Delete it and enter the price again in the new units. Run the price sync once after upgrading, so request fees and cache prices arrive.

lmgw 0.4.1

Tools in voice conversations, llama.cpp as a backend of its own, and an app window that feels native.

  • Tools in voice conversations. A /v1/realtime session can offer MCP tools the way OpenAI's Realtime API does ({"type": "mcp"}): lmgw runs the calls on its own MCP servers, by label, and reports them as mcp_call items; the client sends the follow-up. GET /v1/mcp/servers and /v1/mcp/servers/{label} list the servers and their tools for a client's picker. /v1/responses now takes allowed_tools as {tool_names: [...]} too and refuses a malformed one instead of allowing every tool.
  • llama.cpp gets its own backend. llama-server rows speak their own protocol (existing rows migrate on start): what a model can do comes from the server's /props, and images returned by tools reach a model that sees them. ik_llama.cpp rows use the same path.
  • Configured fallbacks always apply. A fallback that cannot see gets placeholders for the images (a PDF's pages go as its text) instead of the request waiting for the local model, and the response header x-lmgw-images-omitted counts them. A voice turn goes as audio to whichever model answers if it hears audio, cloud and LAN models included (the setting is now off/on, a stored local reads as on); otherwise it goes as the transcript. Voice-clip transcription follows the configured fallback. The Traffic page marks every request whose content was reduced this way.
  • No orphaned model containers. A client that hangs up while its model loads no longer leaves the container running unregistered, which used to block that model with vram_queue_timeout. After start, lmgw adopts or removes the containers it owns, and a request its client gave up on is logged as a 499 row.
  • A native app window on KDE. KWin draws the frame, the app uses the desktop's fonts and the title bar's colours, the traffic pulse and the GPU status sit at the foot of the sidebar, right-click gives a desktop menu instead of a browser's, the window opens fully drawn, and the scrollbars are thin. The default interface scale is now 125%.
  • Browser clients with a gateway key. CORS preflights now echo the requested headers, so a browser can send Authorization.

Download the RPM below and install it with sudo dnf install ./lmgw-*.x86_64.rpm, or let the app offer the update.

lmgw 0.4.0

Voice: talk to your models, in the app and over the API.

  • Realtime voice API. /v1/realtime speaks OpenAI's Realtime protocol over a WebSocket and answers with your own models: turn detection inside lmgw (Silero VAD, and Smart Turn for semantic_vad), speech to text, a chat model with tools, and text to speech. Interrupting cuts the answer, and the history keeps only what was heard.
  • Voice in the Chat. Dictation into the composer, read-aloud of replies (code blocks and tables are announced, not read), and a voice mode that talks with the open thread. Choose the microphone and the output in the app. Each thread can have its own speech models, voice, style and languages: the language you speak and the one the reply is spoken in can differ. A local model that hears audio can get your voice turn as audio (a setting, off by default).
  • Natural speech output. Replies go to text to speech in whole sentences, several to a request, and the silence an engine pads its audio with is capped by a new setting, Settings → Realtime → Longest pause.
  • Audio models. Catalog downloads follow the revision a model's spec pins, a package can be completed in one go, and an audio model can run on the CPU with its own thread count, where it keeps serving while the GPU hold is on.
  • Many fixes around audio containers, the GPU hold and the catalog.

Download the RPM below and install it with sudo dnf install ./lmgw-*.x86_64.rpm, or let the app offer the update. The README has the voice setup.

lmgw 0.3.0

First public release.

lmgw, short for local model gateway, is a Linux desktop app. It runs models on your own GPU and serves them through local copies of the OpenAI and Anthropic APIs. Cloud providers sit behind the same aliases, picked by you or used as a fallback.

Download the RPM below and install it with sudo dnf install ./lmgw-*.x86_64.rpm. Requirements and a quick start are in the README. From now on the app checks this page itself and offers each new release.

Kai

Kai 0.3.0

Latest

Kai 0.3.0 brings Kai into Firefox: with the new Kai for Firefox extension, Kai reads and operates the pages open in Firefox, and Firefox's sidebar runs lmgw's Chat through Kai. It needs lmgw 0.6.0.

  • Kai for Firefox. The RPM ships kai-browser-host, the native- messaging host the extension talks to, and registers it with Firefox. The extension itself is released on its own and signed by Mozilla: https://github.com/lmgw-dev/kai-firefox. Kai takes the link only from a Firefox that Plasma started; Firefox's app scopes in the settings lists the units it accepts.
  • Browser tools. While a Firefox is linked, Kai's tools gain the browser's: tabs, reading (outline, controls, text, tables, PDFs in Firefox's viewer), screenshots for a vision model, pressing, filling and choosing, numbered hints over the controls in view, Firefox's zoom, Reader view, bookmarks, history and downloads, and tools for developers. A PDF's text comes from poppler's pdftotext, run in a bubblewrap sandbox. The tools leave the list when the last Firefox quits.
  • Site rules and asking first. Each site is act, read or none, set by host, domain or origin. Reading runs for you, acting asks first. Kai may act in Firefox without asking (the hands-and-eyes switch) lets Kai act unasked, but only on the tab you brought to the front, on the site where you last acted. Ask before committing still asks before a buy, send or delete, a form that goes to another site or a site your history has never seen. For Kai's own calls, Kai also asks out loud, and the talk key answers with a spoken yes or no.
  • The rules reach the desktop tools. Looking at or reading a Firefox window on a none site is refused, a screenshot blanks such windows grey, and typing into a read or none page is refused.
  • Firefox's own pages and dialogs. Where the desktop's accessibility is on, Kai reads and presses a page's dialogs, a site's permission prompts, the notification bars and Firefox's own pages around the tab in front. It never reads about:logins, the password manager or a password field, and never presses in Firefox's settings or an add-on's prompt.
  • Announcements and reading aloud. Kai can say on its own that a page loaded, a dialog opened or a field shows an error, without a model turn, and read a page aloud paragraph by paragraph with pause, resume and skip. This speech comes from lmgw in Kai's own voice, waits for a quiet moment, and stops when you talk or press the talk key. Announcements have their own switch; turning hands and eyes on from the settings window turns them on too, and you can turn them off.
  • The sidebar is Kai. Kai relays only the Chat's own requests from the sidebar to lmgw, with its device certificate, and tells the sidebar each site's rule.
  • Admin level warning. While a Firefox is linked and Kai's lmgw admin level is read only or full, Kai warns once, and again when the level is raised: a page's text could steer a model to lmgw's configuration.
  • Install it from the settings. The new Firefox section counts the connected Firefox profiles and has Install Kai for Firefox; a build without the install link says how to install it by hand.
  • Settings in plain words. Every setting has a label, a help line, a section, a unit and a fitting control, with Advanced folded away and the key only in a tooltip. Switches are checkboxes, Reset shows only off the default, audio devices and screens are picked from lists, and a footer lists what waits for a restart, with Restart Kai now.
  • Conversation. Choose Shared with your other Kai apps or Only on this computer; the own name starts as "Kai ", and nothing changes until Use this conversation. Listening goes on in the new conversation.
  • When pairing, Kai gives lmgw this computer's name for the Devices card.

Upgrading: Kai 0.3.0 needs lmgw 0.6.0; it is built and tested against that release. The RPM now also requires poppler-utils and mozilla-filesystem and recommends bubblewrap. Requirements are otherwise as for 0.2.0: KDE Plasma 6 on Wayland, PipeWire, KWallet, and lmgw with its device listener on. Install the RPM from this release, or let Kai's update check offer it. For the browser, install Kai for Firefox 0.1.0 (Firefox 154 or newer).

Kai 0.2.0

Kai moves to lmgw's device link: a key of its own, mutual TLS and no bearer. It needs lmgw 0.5.0 or later.

  • Pairing over the device link. A pairing link now carries https URLs, lmgw's fingerprint and a one-time token. Kai makes its own key, enrols it over a connection pinned to that fingerprint, keeps the key in KWallet and connects with a certificate. Kai shows a code, lmgw's Devices card shows the same one, and you click Confirm there. Pasting or clicking the same link again after a failure enrols the same key.
  • One link for everything. The feed, voice sessions, the tool link and the window's requests all use the same mutually authenticated client. Kai tries the link's URLs and then follows the address that answered. It takes the URL list and the dashboard address lmgw reports (Open in lmgw uses it), and the tooltip says when the certificate is due.
  • Refusals in plain words. Kai says whether it waits for your Confirm, the certificate has lapsed or it must pair again. A stray TLS alert from another host on the address never makes Kai drop its pairing.
  • Approvals follow the feed. The approval card shows calls that waited before Kai started, and a call you already decided does not come back.
  • "Always allow" in a tool's confirmation sets that tool to allow for trusted callers.
  • The tray's update check always shows its answer; new app and tray icons.
  • Kai mobile, the Android client, is released alongside: https://github.com/lmgw-dev/kai-mobile
  • MIT licence. Kai was FSL-1.1-MIT.

Upgrading: a pairing from Kai 0.1.0 used a bearer key, which lmgw 0.5.0 no longer accepts. Kai tells you to pair again; do it from the Devices card in lmgw, confirm the code, and Kai removes the old key from KWallet. Requirements are otherwise as for 0.1.0: KDE Plasma 6 on Wayland, PipeWire, KWallet, and lmgw 0.5.0 or later with its device listener on. Install the RPM from this release, or let Kai's update check offer it.

Kai 0.1.0

Kai's first public release: a voice assistant for the KDE Plasma desktop that runs on the models of your own lmgw gateway.

  • Talk from anywhere. A global talk key (tap, or hold for push-to-talk) starts listening in any app; Kai answers out loud, and you can talk over it to interrupt. Everything goes into one ongoing conversation in lmgw's Chat, which you can read and continue in the dashboard. Conversation mode keeps listening for longer exchanges, and the tray switches personality profiles.
  • See what Kai is doing. The tray icon shows idle, listening, thinking, speaking or muted; the ring over the desktop moves with the voice.
  • Desktop tools over MCP. Windows (list, focus, tile, move, close), screenshots and window text for a vision model, the selection, clipboard and files, opening files and apps, running commands, typing and keys, notifications, do not disturb, Night Light, media and volume. look and find hand a question to a second model; long tasks run as background jobs; routines are your own scripted tools.
  • You stay in control. Every tool has a policy (allow, allow if trusted, confirm, deny, off) and a caller level. The approval card shows what a call would do and takes Enter, Esc or a spoken yes or no. Nothing runs while the screen is locked.
  • Overlays and MCP Apps. The model can show text, a sandboxed HTML card, an image, an emote or a weather card over the desktop, and a tool's own MCP Apps view.
  • A settings window for tools, desktop permissions, every setting and personality profiles.

Requirements: KDE Plasma 6 on Wayland (Fedora; the RPM is built on the current release), PipeWire, and lmgw 0.4.3 or later with a paired device key. Install the RPM from this release; Kai checks GitHub for updates and installs only an RPM whose sha256 matches the release's.

Licence: FSL-1.1-MIT, as lmgw.