# viiwork
URL: https://viiwork.pulu.app/
Updated: 2026-10-27

Open source LLM inference for several machines

## Run LLMs across all your GPU machines.

Install one viiwork node on each machine. The nodes find each other, and a request sent to any of them runs on a free slot of whichever machine serves that model, through an OpenAI-compatible API on port 8086.

- [Install a node](/install/)
- [How it works](#how-it-works)

Runs on AMD and NVIDIA GPUs under Linux and on Apple Silicon Macs.

- 8086 the API port on every node
- 4 engines, side by side on one node
- MIT licence, source on GitHub

EXAMPLE MESH (4 NODES): A request runs wherever a slot is free

- ENTRY NODE: OpenAI-compatible API
- NODE 01: AMD GPUs, 2 free slots
- NODE 02: NVIDIA GPUs, 4 free slots
- NODE 03: Apple Silicon Mac, 1 free slot

HOW IT WORKS

## Each machine runs a node, and the nodes pool their slots.

A node serves the models configured on its own machine and learns what every other node serves. A client can send its request to any node and it finds a slot somewhere in the mesh.

### 01 / INSTALL Run the setup wizard on each machine

The wizard finds the GPUs and the GGUF model files, proposes which cards each model gets and how many slots it has, and shows every file before it writes anything.

### 02 / JOIN Nodes find each other

On a Tailscale tailnet a node finds the others through tailscaled, and on a LAN through mDNS, so there is no peer list to keep. A new machine joins a secured mesh with a join code printed by any node.

### 03 / REQUEST Call any node

Every node accepts OpenAI-compatible requests on port 8086. A request runs on a free slot for its model, on the same machine or another one, and waits up to 20 seconds in a queue when every slot is busy.

FEATURES

## A node routes, aliases and measures on its own.

All of this ships in the one viiwork binary and is described in the documentation for the current release.

[Read the documentation](/docs/)

### Several engines on one machine

llama.cpp, vLLM, FreeToken and Strata can run on the same node, each model on its own GPUs. viiwork drives the engines and does no inference itself.

### Routing by measured speed

Each node measures its own time to first token per model and publishes it. Requests are sent mostly to the hosts predicted to answer soonest, and a conversation with a session header stays on one host so its prompt cache stays warm.

### Model aliases

Clients can ask for a stable name such as stable-coder. You change which model it means once, from any machine, and the whole mesh follows.

### Dashboards and viiwork top

Every node serves a page for itself and one for the whole mesh, with models, running requests, backends and prompts. viiwork top shows the mesh in a terminal.

### Power and energy

A node can record wattage, energy per model in kWh and, with a spot price source configured, what the electricity cost.

### Clients that find the models themselves

OpenCode and Roo Code can read the served models and their context lengths from a node, so nobody keeps a model list by hand.

FROM THE BLOG

## Recent posts

[All posts](https://viiwork.pulu.app/blog/)

- [Two RTX 5090 hosts: how much does a faster link help?](https://viiwork.pulu.app/blog/kv-share-two-5090-hosts/): A 1 GbE link was already enough to keep most coding-agent turns warm. A 10 GbE cable made replica transfers faster, but the next bottleneck was elsewhere.
- [Why coding agents need session affinity](https://viiwork.pulu.app/blog/session-affinity-coding-agents/): An overnight agent swarm showed what happens when a conversation keeps moving away from its prompt cache, and why pinning a machine is only part of the answer.

QUICK START

## Ask a node what the mesh serves.

Once the wizard has started a node, list the models of the whole mesh and send a chat completion to one of them. The install guide covers download, verification and adding more machines, and an AI assistant can follow it with you.

[Open the install guide](/install/)

```
# Models served anywhere in the mesh
curl http://localhost:8086/v1/models
# The mesh, live in a terminal
viiwork top
```

Every node accepts requests for the whole mesh