Integrating the mesh API into another application

How to surface a viiwork fleet — models, live jobs, per-GPU load, and the prompt/output history — inside an application you already run, rather than sending people to viiwork's own /mesh dashboard.

Written for an implementor who has not worked on viiwork. It covers the endpoints, the two ways to reach them from a browser application, and the handful of semantics that are not visible in the payloads and will otherwise cost a day each to discover.

Applies to viiwork 2.0 and later. viiwork 2 changed the shape of every payload in section 4 — one node per machine, one port, and a membership list where version 1 had a node plus a list of polled peers. If you are reading a local / peers structure, you are talking to a 1.x node and want that version's document.

1. What you are building against

viiwork is an LLM inference mesh. One node runs per machine, supervising every model configured there, and serves its API and dashboards on port 8086. Nodes find each other by themselves — over a tailnet, by mDNS on a LAN, or from a seed address — so there is no peer list to maintain anywhere.

Any node is a complete entry point: ask one node and it reports the whole mesh. You do not need to reach every machine, and you do not need to know the topology in advance. One HTTP call to one reachable node gives you the fleet; one held-open stream gives you live activity for all of it.

Requests follow free slots rather than round-robin: a node runs a request locally if it has a free slot, otherwise forwards to the member with the most free slots, otherwise queues it. A caller does not choose a host, and should not try to.

Most of what the API exposes is in-memory state that a restart clears — activity, prompt history, metrics. Two things are durable and survive a restart: the alias table in the node's state directory, and the energy store on hosts that enable it.

2. Access and trust

Inference and read endpoints are unauthenticated. No API keys, no tokens, no sessions. Anything that can open a TCP connection to port 8086 can list models, read prompt history and send a completion. This is deliberate — viiwork expects a private network, where reachability is the authorization model — but it has consequences you must design around:

Whatever you build must not become a path from an untrusted network to a node. If your application is reachable more widely than viiwork is, your application's own authentication is the only thing in between. Put the integration behind it.
Prompt and output text are readable over the API, unauthenticated like everything else. If the traffic on your fleet is sensitive, that is a reason to keep the integration server-side.

Three things are not open, and they are the exceptions to remember:

Mesh membership is either secured — a shared 32-byte secret, encrypted gossip and signed forwards — or explicitly open. mesh on /v1/cluster tells you which. This protects the mesh from joining, not the API from being read.
Alias writes (PUT/DELETE on /v1/aliases/<name>) need a signature from that secret in a secured mesh, and must come from the node's own machine in an open one. Reading the alias table is open like everything else.
Chassis power control acts only on hosts in the node's configured allowlist, published as power_control.hosts. There is no wildcard.

How your application reaches the fleet — VPN, overlay network, routed LAN, service discovery — is deployment-specific and out of scope here. This document assumes only that some host running your code can make HTTP requests to a node.

3. Two integration modes

PATH A — SERVER-SIDE PROXY (RECOMMENDED)

  Browser  ──same origin──▶  Your server  ──private network──▶  viiwork node

  No CORS involved · your existing auth applies · works against any version

PATH B — DIRECT FROM THE BROWSER (FALLBACK)

  Browser  ───────────private network, needs CORS───────────▶  viiwork node

  Viewer's browser must reach the node itself · needs CORS · no auth gate

The asymmetry that decides it: your server can almost certainly reach the fleet; your users' browsers may not.

Path A — server-side proxy

Add endpoints in your own application that fetch from a node and return the response. Two properties make this the default choice:

Your application's authentication becomes the gate. viiwork provides none. A proxy endpoint inside your authenticated area is the only arrangement where fleet access is actually controlled.
The viewer does not have to be able to reach the fleet. With Path B, a user outside the private network sees an empty page and no useful error.

Streaming passes through unchanged in any runtime with a streaming HTTP client — forward the upstream response body rather than buffering it, or the live view will arrive all at once when the connection closes.

Put the node's base URL in configuration (an environment variable), not in source. Nodes move.

Path B — direct from the browser

Needs the node's CORS allowlist to name your origin. Take it only when Path A genuinely cannot do the job — for example an event stream you would rather not hold open through your own server — and accept that you are giving up the auth gate.

Configure the allowlist under api.cors in viiwork.yaml:

api:
  cors:
    allow_origins: ["*.tail1234.ts.net", "localhost", "127.0.0.1", "*.your-app.example"]
    allow_tailnet_ips: true

*.example.com matches subdomains only, never the bare apex; every other entry must match the host exactly. Left out, the list is derived from tailscaled: this tailnet's own MagicDNS domain, localhost, and literal tailnet IP ranges. A list you write replaces that, so include your tailnet's domain in it — never *.ts.net, which is every Tailscale customer's — and your application's own origin, which is deployment-specific and you must add. allow_origins: [] sends no CORS header at all. Writes (POST and friends) from an origin the list refuses are answered 403; see security.md.

Preflights are answered, and a refused origin gets 403 rather than a silent failure, so a misconfiguration is visible in devtools. Cross-origin responses expose X-Gpu-Backend, X-Viiwork-Node, X-Viiwork-Origin, X-Viiwork-Queued-Ms, X-Viiwork-Alias and X-Viiwork-Model.

Do not mix modes. Pick one per data source and stay there. An integration that proxies its snapshots but opens its event stream directly will silently show live data only to viewers whose browsers happen to reach the fleet, and the difference is invisible until someone reports "the job list is always empty" from a machine you cannot see.

4. API reference

Every endpoint below is GET, returns JSON, and requires no headers.

`GET /v1/cluster` — the primary call

Whole-mesh snapshot: every member the node knows, each with the status it last reported. This is the one endpoint a fleet overview needs.

{
  "view": "node-a",
  "mesh": "secured",
  "models": ["some-model-27B", "another-model-8b"],
  "members": [
    {
      "node": "node-a",
      "addr": "198.51.100.10",
      "role": "node",
      "state": "alive",
      "status": {
        "node": "node-a",
        "node_id": "viiwork-4f133f9145e6a4eb",
        "ver": "v2.0.0",
        "addr": "198.51.100.10",
        "api_port": 8086,
        "uptime_s": 3312,
        "host_mem_total_mb": 64198,
        "host_mem_used_mb": 16848,
        "prompt_history": 1000,
        "energy_kwh_24h": 15.9,
        "energy_kwh_30d": 169.0,
        "power":  { "watts": 978, "available": true, "source": "dcmi" },
        "cost":   { "available": false },
        "gpus":   [ { "gpu_id": 0, "util": 74, "vram_used_mb": 12568, "vram_total_mb": 16368, "power_w": 148 } ],
        "models": [ {
          "name": "some-model-27B", "engine": "llamacpp",
          "engine_name": "llama.cpp", "engine_version": "b11804",
          "slots": 2, "busy": 1, "queued": 0, "ctx": 49152,
          "backends": [ {
            "id": "some-model-27B/0", "status": "healthy",
            "gpus": [0, 1], "pid": 15, "rss_mb": 10547,
            "slots": 2, "busy": 1, "respawns": 0, "uptime_s": 3298
          } ]
        } ]
      }
    }
  ],
  "power_control": { "hosts": ["node-a", "node-b"] }
}

Field	Meaning
`view`	The node that answered. Everything else is its view of the mesh.
`mesh`	`secured` or `open`. See §2 — it describes who may join, not who may read.
`models`	Every model served anywhere in the mesh, deduplicated. The quick answer to "what can I ask for".
`members[].node`	The member's name, which is its hostname unless configured otherwise. This is the grouping key for anything measured per machine — it is a real name, not derived from the address it was dialled on.
`members[].state`	`alive` · `dead` · `left`. A node that stops gracefully goes `left`; one that vanishes goes `dead` after the mesh's suspicion window. `meshapi` also defines `suspect` for other implementations; viiwork never reports it. Keep showing a dead member rather than hiding it.
`members[].role`	`node` (serves models) or `gateway` (routes only, no models, null status).
`members[].status`	The member's last `/v1/status`, or `null` for a gateway or a member that has not answered yet. Always null-check it.
`status.ver`	Build string of that machine. Mixed versions are normal during a rollout.
`status.models[]`	One entry per configured model, with its aggregate `slots` / `busy` / `queued` and `ctx` per slot.
`status.models[].engine`, `engine_name`, `engine_version`	`engine` is the stable id (`llamacpp`, `vllm`, `freetoken`, `strata`): key on it. `engine_name` is the name to show (`llama.cpp`, `vLLM`), present from v2.9.2; on an older node show `engine`. `engine_version` is the engine's own version on that machine, a version string and nothing else. It is absent for an engine that reports none (FreeToken, Strata), when it could not be read, and for the first moments after a node starts. Absent is "cannot say". Both fields reach you only when the node that answers is itself on v2.9.2 or later: an older node relaying a newer member's status drops them.
`status.models[].backends[]`	The processes behind it. `gpus` is the list of cards one backend holds — two or more means a tensor-split group. `status` is `starting` · `healthy` · `unhealthy` · `dead`; cold-loading a large model sits in `starting` for minutes.
`status.power`	`{watts, available, source}`. When `available` is false, ignore `watts` rather than rendering a zero. `source` is `dcmi`, `sdr`, `sensor:<NAME>`, `nvidia-smi` or `rocm-smi` — diagnostic, for when an operator asks why a host reads what it does.
`status.energy_kwh_24h`, `energy_kwh_30d`	Rolling whole-machine energy from the durable store, present only where it is enabled. Absent is "not recording", not zero.
`status.client_energy`	Energy per named client on this node: `since`, `through` and `clients[]` with `name`, `wh_total`, `w_1m`, `kwh_24h`, `kwh_30d`, `busy_s_total`, `tokens_total`. Present only on a node with the energy store that has served a named client. Counters per node: do not sum them across nodes. See `docs/client-energy.md`.
`status.host_mem_*`	Host RAM, `used` being `MemTotal - MemAvailable`, so page cache is not counted as pressure. Coarsened on the pushed stream — see §6.
`power_control.hosts`	Machines this node may power-control. Rows for the fleet should be the union of members and this list: a powered-off machine is absent from the mesh entirely, and this is the only place its name still exists.

Everything per-machine is reported once, because there is one node per machine. Version 1 needed a deduplication rule here; version 2 does not.

`GET /v1/capacity`

What the router polls, once a second, to decide where a request goes. Cheaper than /v1/cluster and the honest answer to "is there room right now".

{
  "node": "node-a", "ver": "v2.0.0",
  "models": [ { "name": "some-model-27B", "engine": "llamacpp",
                "slots": 2, "busy": 0, "queued": 0, "ctx": 49152,
                "backends": 1, "healthy_backends": 1 } ]
}

Free capacity is slots - busy, and healthy_backends of 0 means the model is listed but can serve nothing. A model appears here whatever state its backends are in, so check both.

A model the node has measured also carries its performance score (v2.7.0): ttft_overhead_ms, prefill_ms_per_1k (milliseconds to read 1,000 prompt tokens) and perf_samples. All three are absent when the node cannot say — no measurement yet, an older version, or routing.performance: false — and absent never means zero.

A model also carries images (v2.10.0): true when the node's backends of the model read images, false when they do not. It is absent when the node cannot say: an engine that does not report it, backends that are not healthy yet, or an older version. Absent is not false. The same field is on each model of /v1/status and /v1/cluster, and on /v1/fleet/capacity, where a model's images is true when some host with slots says yes and false only when every such host says no.

`GET /v1/aliases`

The mesh-wide alias table. Aliases give clients a stable name for a model whose real id carries a quant and a version.

{ "aliases": [ {
  "name": "stable-coder", "target": "some-model-27B", "fallbacks": [],
  "resolved": "some-model-27B", "state": "ok",
  "updated_at": "2026-09-12T15:10:26.992Z", "updated_by": "node-a"
} ] }

Field	Meaning
`state`	`ok` — resolves to its target · `fallback` — the target cannot serve, a fallback is answering · `unavailable` — nothing can serve it, requests get 503 · `shadowed` — a real model now has this name, which wins.
`resolved`	What a request for `name` will actually run on, or `null` when nothing can.
`fallbacks`	Tried in order when the target cannot serve.

Reading is open; writing needs a signature (§2). An alias may be pre-staged against a model no one serves yet — it reads unavailable until that model appears, then starts resolving on its own.

`GET /v1/status`

The asked node only. It is exactly the status object shown inside each member above — same fields, same meanings — and it is what members poll each other with. Use it when you are addressing one specific machine; otherwise /v1/cluster already contains it for every member.

Two fields on a model are newer than the example above and absent on older nodes: parked: true for a model taken down with viiwork down (it has no backends until viiwork up), and perf, the node's measured score for it (overhead_ms, prefill_ms_per_1k, samples, baseline_age_s).

`POST /v1/models/down`, `POST /v1/models/up` — park a node's models

What viiwork down and up send: the node stops (or starts again) the named models and stays in the mesh. Writing needs a signature, as for aliases (§2).

{ "models": ["some-model-27B"] }

{} or no body means every model the node is configured with. A body with other keys and no models is refused with 400, so a misspelt key cannot park everything. The answer lists each model with parked and changed; an unknown name is 400 and changes nothing.

`POST /v1/power`, `POST /v1/mesh/power` — chassis power

Both take {"host": "...", "action": "status|on|off|cycle"} and return {"host", "action", "result", "via"}, or {"error"}. Disabled by default; a node without it configured answers 503, not 404, so you can tell "will not" from "does not know how".

/v1/mesh/power is the one to call. It forwards to the node living on the target host, and reaches the host's BMC over the network only when nobody answers there — which is the only way to act on a host that is powered off. /v1/power is the executor and acts solely on the node you address it to.

Refusal	Meaning
403, "not listed in power.control.hosts"	The host is not in the allowlist. There is no wildcard; being a peer is not enough.
403, "is serving this dashboard"	A node will not power off its own host — that would destroy the answer to the request. Ask a different node. `status` is still allowed.
400	Unknown action, or `/v1/power` addressed at some other host.
502	The host is unreachable and has no out-of-band BMC configured, so it cannot be woken.

There is no authentication on this, as with the rest of the API. Anything that can reach a node can power off any host in that node's allowlist. Keep the allowlist to the hosts you actually want reachable this way, and prefer a server-side proxy (Path A) if a browser needs to drive it.

power_control on /v1/cluster lists what a node will accept — hosts, and out_of_band for the subset reachable while off. Hosts in that list may be absent from members entirely: that is what a powered-off machine looks like, and it is why the list is published at all.

`GET /v1/models`

OpenAI-shaped model list, aggregated across the whole mesh, with owned_by saying where each name comes from:

{"object": "list", "data": [
  {"id": "some-model-27B", "object": "model", "owned_by": "local",
   "max_model_len": 32768, "context_length": 32768},
  {"id": "another-model-8b", "object": "model", "owned_by": "peer",
   "max_model_len": 98304, "context_length": 98304},
  {"id": "stable-coder", "object": "model", "owned_by": "alias",
   "max_model_len": 98304, "context_length": 98304}
]}

local is served by the node you asked, peer by another member, alias is a name from the alias table, and pipeline a configured multi-step chain. A client can send any of them as model — the distinction is for display, not for routing. Aliases are listed whatever their state, so a name here is not a promise it can serve right now; /v1/aliases carries that.

max_model_len and context_length carry the same number in two spellings — vLLM's and OpenRouter's — because OpenAI never extended this endpoint with a window and the ecosystem converged on those two names instead. The number is the served context: the total prompt+completion budget per slot that the fleet guarantees, which is the minimum across fresh hosts that have slots, not a checkpoint's architectural maximum. Both are omitted when no host can say — absent is not zero — and a node older than v2.3.0 omits them always. There is deliberately no output ceiling here: viiwork has none distinct from the shared window. See autodiscovery.md, which also covers the two other shapes the same figure is published in.

`GET /health`

Container-style health for the node you ask, shaped for a Docker healthcheck:

{"status": "ok", "version": "v2.0.0", "node": "node-a",
 "uptime_seconds": 3312, "backends_healthy": 6, "backends_total": 6}

200 when at least one backend is healthy, or when the machine has no models configured at all — a pure router is a valid node. 503 with "status": "unhealthy" when models are configured and not one backend can serve, which includes the whole cold-load window after a restart and a node still joining the mesh.

It answers for one machine, so use it as a reachability and readiness signal, not as fleet health: /v1/cluster is what tells you the mesh is well.

`GET /v1/mesh/stream` — `text/event-stream`

One held-open connection carrying two named event types. Nothing polls: cluster snapshots are diffed server-side and pushed only when they change, so an idle mesh is silent. Peer fan-out happens on the server, so one reachable node is enough to see the whole mesh.

event: activity
data: {"t":1756360000,"type":"request","message":"some-model → ts-0,1","rid":41,
       "task_id":"","gpu_id":-1,"node_id":"viiwork-50f3…","hostname":"node0","addr":""}

event: cluster
data: { …a full /v1/cluster payload… }

Handle them by name — es.addEventListener("activity", …) and ("cluster", …). A bare onmessage handler receives nothing, because named events never dispatch to it.

Activity field	Meaning
`t`	Unix seconds.
`type`	`request` · `backend` · `system`. Only `request` events carry a `rid`.
`message`	Human text, shaped `"<model> → <destination>"`, with `" done (1.2s)"` or `" aborted by client"` appended on completion. Split on `→` rather than re-deriving the parts.
`rid`	Request id. Per-process counter, not cluster-wide — see §6.
`node_id`, `hostname`, `addr`	Which member the event came from. `addr` is empty for the node you are connected to and set for other members; you need it for prompt lookups.

`GET /v1/activity`, `GET /v1/activity/stream`

Single-node event log — the last 200 events, and an SSE stream of the same. No peer merging and no server-side fan-out, so for a mesh view prefer /v1/mesh/stream. Useful when you are deliberately looking at one node.

`GET /v1/metrics`, `GET /v1/metrics/stream`

Per-GPU utilisation and VRAM history — one hour at 5-second resolution (720 samples per GPU), plus a live stream. Returns {"available": false} where GPU metrics tooling is missing; handle that rather than assuming samples exist.

`POST /v1/chat/completions`

Standard OpenAI chat completions, streaming or not, routed to whichever member has a free slot for the requested model. /v1/completions and /v1/embeddings share the same routing, headers and errors. Relevant only if your integration includes a prompt box.

The model may be a real model id, an alias, or a pipeline name — /v1/models lists all three and the caller does not need to know which it has.

Response headers say what actually happened:

Header	Meaning
`X-Viiwork-Node`	The member that ran it. Always present.
`X-Viiwork-Origin`	The member that received it, present only when the request was forwarded. Absent means the node you asked ran it itself.
`X-Gpu-Backend`	The backend id that executed it, `<model>/<n>`.
`X-Viiwork-Queued-Ms`	Milliseconds spent in the origin queue, present only when it waited.
`X-Viiwork-Alias`, `X-Viiwork-Model`	Present when an alias was resolved: the name asked for, and the real model that answered.

Errors: 429 means every member is at capacity and the queue could not take it — respect Retry-After. 503 means nothing can serve the model, which is also what an alias whose target and fallbacks are all unavailable returns; it is also the answer, with Retry-After, when every route a request tried refused it as busy or unreachable before responding. 502 means a route failed the request in some other way. 404 means the model is unknown to the mesh entirely.

A response is never retried once its first byte has gone out. If the engine or peer serving it fails after that, the node drops the connection rather than ending the response normally, so a truncated stream shows up as a read error (no [DONE], no final chunk) and never as a short but complete answer.

A queue sits in front of this, so a brief burst above capacity waits rather than failing: routing.queue_timeout is 20 s by default. Treat 429 as real backpressure, not as a retry-immediately signal.

Keeping a session on one host. Send X-Session-Affinity (or X-Session-Id) with a value that is stable for one conversation, and its turns go to the same host and backend while that host is among the fastest with a free slot, from whichever node they enter. That is what lets a long conversation reuse the engine's prompt cache instead of reading the whole history again on another machine.

Pinning a host. ?host=<hostname> on any of the three inference endpoints narrows routing to one machine — POST /v1/chat/completions?host=node-b. Absent or mesh routes as usual. The name is a member name (node-b), not host:port, and it is matched against members the node already knows. A host that is not serving the model answers 404 with a message naming both, never a silent fallback to the mesh, and a malformed value is 400. X-Viiwork-Origin on the response names the peer the request was proxied to; its absence means the answering node ran it. The value is only compared against hostnames the node already knows and is never dialled, so it cannot reach anything normal routing could not. Through the gateway a pinned request may take two hops (gateway → a node serving the model → the pinned host); that is expected and invisible to the caller.

Preferring hosts. ?prefer=node-a,node-b names machines in order of preference: the first one with a free slot for the model runs the request, and when none has room the request goes wherever it would have gone without the list. It is the soft form of the pin: an unknown name or a powered-off host is passed over rather than answered with 404, and only a malformed value, or more than eight names, is 400. A client that cannot add a query parameter (most OpenAI SDKs append the path to a base URL) sends the same list as the X-Viiwork-Prefer header; the parameter wins when both are present, and ?host= wins over both. A preferred host is taken even when the mesh has measured another as faster, and even when the request's session lives elsewhere, so a session that spilled to another host while the preferred one was full moves back on its next turn and reads its prompt again there. X-Viiwork-Node on the response names the machine that ran it.

Images. A message's content may hold image_url parts, as in the OpenAI API, with a data: URL or a link the engine can fetch. The body passes to the engine unchanged and may be 32 MB at most. From v2.10.0 such a request is routed only to a host that reads images with the model; see models.md. When every host that serves the model says it reads none, the answer is immediate:

{"error": {"type": "invalid_request", "message": "model \"some-model\" does not read images on any node"}}

with status 400. With ?host= naming a node that reads no images the message names that host instead: model "some-model" does not read images on host "node-b" (v2.10.1). Check a model's images on /v1/fleet/capacity, or its inputs in /api.json, before offering an upload.

X-Viiwork-Client is a mesh header. A node removes it from a client's request; a client is named by its source address only. That holds once every node clients call runs v2.10 or later; a v2.9.x node passes the header on.

5. Prompt and output history

The part of the API with real sharp edges. Read this before designing a request list.

Every proxied request has its prompt captured when it starts and its output captured when it finishes, keyed by request id. Text is never pushed on the event stream — the stream carries only that a request happened — so the list of requests and the content of any one request are two separate fetches. That is deliberate: full bodies on every event would put per-request payload on a path most consumers never look at.

`GET /v1/mesh/prompt?rid=<id>&addr=<host:port>` — use this one

Fetches one stored request. addr is the addr from the activity event that produced the rid; pass it empty for a request the asked node ran itself. The node forwards to the named peer server-side and returns the result, which is what makes lookups work when the caller cannot reach that peer directly.

{
  "rid": 41,
  "t": 1756360000,
  "model": "some-model-27B-Q4_K_XL",
  "prompt": "what is 2+2?",
  "output": "2 + 2 is 4.",
  "elapsed_ms": 1240
}

Field	Notes
`prompt`	Last user message, or the raw `prompt` field for legacy completions. Can be absent — a multimodal request with array content parts yields none.
`output`	Absent while the request is still running, present once it finishes. For a failed request this is the error body, which is usually the most useful thing to see.
`elapsed_ms`	Wall time, recorded with the output. Absent until then.
`output_tokens`	The reply's `usage.completion_tokens` (v2.8.0). Absent when the node keeping the entry saw no usage object — never a measured zero.
`gen_ms`	Milliseconds from the first token to the end of the reply (v2.8.0); `output_tokens` over this is the average rate. Absent for a non-streamed reply and for a request another node executed.

Reasoning models. With thinking enabled a model puts its answer in a separate reasoning channel. output then arrives labelled — "[reasoning]\n…\n\n[answer]\n…", or "[reasoning]\n…" alone when there was no separate answer. Render those sections distinctly rather than as one blob; the labels exist precisely so a consumer can split them.

`GET /v1/prompts?rid=<id>`

Same payload, the asked node's own store only, no forwarding. This is what /v1/mesh/prompt calls on the owning peer. Use it only when you are certain which node minted the id; otherwise it 404s.

Security constraint — do not work around. /v1/mesh/prompt forwards only to addresses already in the answering node's configured peer list. That check is load-bearing: without it the endpoint fetches whatever address it is handed and echoes the response back, which is a server-side request forgery primitive on a private network. If a lookup returns 400 unknown peer, the fix is the peer configuration on the viiwork side — never a proxy in your application that fetches arbitrary addresses on a caller's behalf.

Storage limits — design the UI around these

Capacity — the last activity.prompt_history requests per node, 1000 by default, evicted oldest-first. It is per-node configuration, so different nodes in one mesh may keep different depths. Each node reports its own value as prompt_history on /v1/status, and so in each member's status on /v1/cluster. Read an absent value as "unknown", never as zero. Size any client-side list from the largest value you see rather than hardcoding one, or you will evict rows the owning node could still answer a lookup for.
Durability — memory only. A node restart loses everything, with no signal other than lookups starting to 404.
Truncation — prompt and output are each capped at 50 000 characters, suffixed "... [truncated]".
Failure mode — a lookup for an evicted or restarted-away id returns 404. Treat it as expected, not as an error: "this request has aged out of the node's history".

6. Semantics that bite

Five properties that are not visible in the payloads.

Request ids are per-node, not cluster-wide

rid is a process-local counter. Two nodes will both mint rid 41 for unrelated requests within seconds of each other. Key every request by (node_id, rid) and carry the event's addr alongside, or your list will merge unrelated work and open the wrong prompt.

There is no server-side registry of running jobs

No endpoint returns "what is running now". In-flight requests are reconstructed by the consumer from the event stream: a request event adds an entry, a matching event whose message contains done or aborted removes it, keyed as above.

Both streams replay recent history when the connection opens, which repairs most of what that costs: every event from the last 30 seconds, plus the events of any request in the ring that has not finished yet, however long ago it started. Replayed events carry "replay": true and arrive before live ones, so a consumer that connects mid-flight sees the jobs already running, and one that reconnects and rebuilds (below) ends up with exactly the requests still running, however long it was away. Older history is not replayed; a consumer that needs it reads /v1/activity.

Two obligations come with it:

Rebuild on reconnect, do not carry state across. Anything that finished during a gap leaves a start with no done — a row that ages forever. Clear the reconstructed set when the stream opens and let the replay repopulate it. A request older than the node's ring is then dropped rather than re-added, which under-reports rather than inventing work; the aggregate counts stay correct either way.
Deduplicate anything you display. Reconstructed state is keyed and so replays for free, but an append-only event log will show every replayed line a second time. Key on node, request id, timestamp and message, and hold the set deeper than the list you render.

The ring bounds it: a gap longer than maxEvents on a given node cannot be repaired from the replay. The aggregate counts in /v1/cluster (in_flight per backend) remain correct regardless and are what to trust for totals.

Freshness differs by column

Data	Freshness	Source
Other members' jobs / activity	live	merged event streams
Local backend + GPU state	~5 s	health-check tick
Members' slots, busy and queued	~1 s	`mesh.capacity_poll`
Members' full status, GPU load	~5 s	member status poll

Do not present all three as equally current. A job appearing instantly next to a GPU utilisation figure that is ten seconds stale reads as a contradiction.

client_energy changes once a minute and is 60 to 90 seconds behind the clock, like energy_kwh_24h. It adds no pushed snapshot of its own. Ignore a node's w_1m when its through is more than about three minutes old. "No client_energy anywhere" can also mean the node you poll /v1/cluster through is on an older version: it drops fields it does not know when it re-encodes member status.

Host memory is coarsened on pushed snapshots

Snapshots on /v1/mesh/stream are diffed server-side and sent only when they change. Host memory ticks every second on a live host — measured at ~86 MB/s, spiking past 600 MB — and an exact figure defeats that entirely, so it used to be removed from the pushed copy altogether.

It is now included but coarsened: quantized to 64 buckets of the host's total, and held at the published level until the reading drifts a full step away from it. In practice the value moves only when memory moves visibly, and a host that is merely idling republishes nothing. Two consequences for a consumer:

Treat host_mem_used_mb from the stream as approximate, within about one bucket (~1 GB on a 64 GB host). Render it as a level or a percentage, not as an exact figure, and never diff two stream values to infer an allocation.
/v1/cluster remains exact. Call it directly if you need the real number — for an alert threshold, a capacity report, or anything a human will quote.

SSE, not WebSockets

The feed is one-way and viiwork carries no WebSocket dependency. Use EventSource (Path B) or forward the stream through your own server (Path A). Note that EventSource sends no preflight — under Path B it relies on the allow header being present on the GET response itself, which it is.

7. Interface expectations

This document specifies no visual design: build with your application's existing components and conventions so the result is indistinguishable from the pages around it. Do not port viiwork's dashboard styling across.

Two behavioural requirements, because both are load-bearing rather than cosmetic:

A freeze control on any live list. Rows in a live-updating request list move under the pointer, which makes them effectively unclickable during busy periods. Freezing must queue incoming events and apply them on resume, not merely stop re-rendering — a frozen render over a still-mutating store leaves rows on screen whose entries have already been evicted underneath.
Request detail opens as a real link. The working pattern is triaging many requests at once, so a request row should be a link to its own URL rather than a click handler opening a dialog. Cmd-click, middle-click and "open in new tab" all need to work, and each tab needs a title identifying which request it is.

Everything else — layout, grouping, which columns to show, whether GPU load gets a meter or a number — is yours. viiwork's own /mesh page is a reference for what information matters, not a design to copy.

8. Before you start

Check what the node is actually running. Every member's build is in status.ver on /v1/cluster. A 1.x node answers a completely different shape — local and peers instead of members — so if those keys are present you are on the wrong document, not looking at a bug.

Expect a mixed fleet during a rolling upgrade. Machines are converted one at a time, so several versions of the contract are live at once. Wire fields are additive and omitted when a node cannot say, so handle a field's absence rather than its emptiness, and never read a missing number as zero.

Cold loads are long. A large model loading from network storage can hold a backend in starting for twenty minutes or more, during which the node answers /health but serves no traffic. Any "is the fleet up?" indicator must distinguish node reachable from backends healthy, or it will report a green fleet that cannot take a request.

Stale peer entries are normal. A peer list is static configuration. Nodes that moved, changed port, or were retired stay in /v1/cluster as unreachable, and prompt lookups against them fail with unknown peer. Render that honestly; it is a fleet configuration matter, not something your integration should paper over.

9. Acceptance checklist

Verifiable against a fleet under load.

The whole mesh renders — every node, its models, healthy backend counts, per-GPU utilisation and VRAM — from a single call to one node.
A member that is dead or left still appears, in that state, rather than being omitted or shown as healthy.
A cold-loading backend reads as starting, not as up.
In-flight requests appear and clear live, with model, destination and elapsed time.
Recent requests list correctly across nodes, with no id collisions.
Opening a request shows its prompt and, once finished, its output; reasoning and answer are visually distinct where both are present.
Opening several requests at once puts each in its own identifiable tab.
Freezing a live list stops all movement, and resuming applies everything that arrived meanwhile, in order.
A lookup for an aged-out request reads as history expiry, not as an error.
Every call is behind your application's own authentication.
A user whose browser cannot reach the fleet directly sees the same thing as one whose browser can.

Reference implementation of everything above: viiwork's own /mesh and /prompt pages, plus the "Mesh Dashboard", "Prompt and output history", "Security" and "API Endpoints" sections of the README.

1. What you are building against

2. Access and trust

3. Two integration modes

Path A — server-side proxy

Path B — direct from the browser

4. API reference

GET /v1/cluster — the primary call

GET /v1/capacity

GET /v1/aliases

GET /v1/status

POST /v1/models/down, POST /v1/models/up — park a node's models

POST /v1/power, POST /v1/mesh/power — chassis power

GET /v1/models

GET /health

GET /v1/mesh/stream — text/event-stream

GET /v1/activity, GET /v1/activity/stream

GET /v1/metrics, GET /v1/metrics/stream

POST /v1/chat/completions