Install viiwork

This guide installs viiwork v2.11.1, the release the documentation on this site describes. If an AI assistant is helping you, give it this page's address and ask it to go one step at a time. A join code is your mesh secret: type it into the wizard yourself and keep it out of the chat.

Before you start

Every machine needs GGUF model files in one directory; the wizard does not download models. Port 8086/tcp (the API and dashboards) and port 7946/tcp+udp (mesh membership) must be free on the machine and open between the machines of the mesh. With more than one machine, put them all on the same Tailscale tailnet or on the same LAN.

NVIDIA on Linux

The driver works (nvidia-smi lists the card), Docker Engine has the Compose v2 plugin (docker compose version works), and the NVIDIA Container Toolkit is installed. With Docker 25 or newer its CDI spec is enough. The wizard writes the configuration and starts the node.

Radeon VII, MI50 or MI60 on Linux

These are gfx906 cards, the hardware viiwork was first built for. The in-kernel amdgpu driver must be loaded (/dev/kfd exists); ROCm is not needed on the host, because it is inside the image. You also need Docker Engine with the Compose v2 plugin. There is no published image for gfx906, so the wizard writes the configuration and you build and start the image yourself, as shown in Build the image on gfx906. The wizard also finds other AMD cards, but no image is verified for them yet.

Apple Silicon Mac

Run the wizard in Terminal on the Mac itself while you are logged in. The node runs natively as a LaunchAgent in your login session, so an SSH session with nobody logged in is refused. There is no Docker on a Mac: the wizard fetches the llama.cpp build its release is pinned to and checks it against the sha256 built into the signed viiwork binary.

viiwork authenticates nothing on its API: anyone who can reach port 8086 can use it, and can read the recent prompts and responses. Keep the machines on a tailnet or another network you trust, or put a firewall or a reverse proxy in front. The security page explains the trust model.

Download and check the release

Set os to the machine's platform: linux_amd64, linux_arm64 or darwin_arm64. The commands download the release archive and its SHA256SUMS from GitHub, check the archive, and unpack it.

v=v2.11.1
os=linux_amd64               # or linux_arm64, darwin_arm64
base=https://github.com/janit/viiwork/releases/download/$v
curl -fLO "$base/viiwork_${v}_${os}.tar.gz" -fLO "$base/SHA256SUMS"
sha256sum --check --ignore-missing SHA256SUMS
# on a Mac instead: shasum -a 256 --check --ignore-missing SHA256SUMS
tar xzf "viiwork_${v}_${os}.tar.gz" && cd "viiwork_${v}_${os}"

Go on only if the check prints OK for the archive. The checksum file is signed as well; checking a download by hand shows how to verify the signature. A newer release may exist on the releases page; its own documentation is in the repository at its tag.

Run the setup wizard

sudo ./viiwork init          # Linux
./viiwork init               # Apple Silicon Mac, without sudo

The wizard checks the machine, lists the GPUs, and asks for your models directory. For each model it finds, you choose whether to serve it and what to call it. Use the name the rest of your mesh uses for that model, so its requests can run on any machine. It then proposes which cards each model gets, the context per slot and the number of slots; you can accept, edit one model's line or drop a model.

Next it looks for nodes on your tailnet and LAN and asks whether to join an existing mesh with a join code, start a new secured mesh, or start an open mesh without a secret. It asks whether this machine accepts mesh-wide updates and what to call the node, then shows every file it will write, with the secret masked. Nothing is written until you say yes. On NVIDIA and on a Mac it then starts the node, waits for every model to load and runs its checks. Pressing Ctrl-C at any question leaves nothing behind.

Build the image on gfx906

On a Radeon VII, MI50 or MI60 the wizard stops after writing /etc/viiwork. Build the image from a checkout of the same release, point the compose file at your models directory and start it:

git clone https://github.com/janit/viiwork && cd viiwork && git checkout v2.11.1
make docker
sudo cp configs/docker-compose.v2.example.yaml /etc/viiwork/docker-compose.yaml
sudo sed -i 's#- /models:/models:ro#- /path/to/models:/models:ro#' /etc/viiwork/docker-compose.yaml
sudo docker compose -f /etc/viiwork/docker-compose.yaml -p viiwork up -d

Replace /path/to/models with your models directory before you run the sed line. make docker needs the ROCm base images; BUILDS.md lists them. The setup page has the details under when it writes the config only.

Check the node

Ask the node for its models, then send a chat completion to one of the names it returns:

curl http://localhost:8086/v1/models
curl http://localhost:8086/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model":"<name from /v1/models>","messages":[{"role":"user","content":"Hello"}]}'

The dashboard for the node is at http://localhost:8086/ and the one for the whole mesh at http://localhost:8086/mesh. viiwork top shows the mesh in a terminal. On a Mac the binary is installed in ~/.local/bin, which is not on the default PATH; the Mac section of the setup page shows the line to add.

Add another machine

On any node of a secured mesh, print a join code:

sudo sh -c 'set -a; . /etc/viiwork/mesh.env; /usr/local/bin/viiwork join-code'   # Linux
~/.local/bin/viiwork join-code                                                    # Mac

The code starts with viiwork1- and is the mesh secret itself, so treat it like a password. Run the wizard on the new machine and paste the code at the mesh question. The code also carries the first node's address, so the new machine must be able to reach it. In an open mesh, viiwork join-code --open prints a code without a secret. The mesh page covers discovery, the two mesh modes and routing.

If the node does not come up

The wizard keeps its files and prints each backend that is not healthy, followed by the node's last log lines. The full log is here:

sudo docker logs viiwork                       # Linux
tail -f ~/Library/Logs/viiwork/viiwork.log     # Mac

The usual cause is a model that does not fit at the chosen context: lower its context or give it more gpus in the config. A model whose chat template llama.cpp cannot parse logs chat template parsing error and needs flags in its args. After editing the config, reload the node; it restarts only the models whose entries changed:

sudo docker kill -s HUP viiwork                                   # Linux
launchctl kill HUP gui/$(id -u)/fi.viiwork.node                   # Mac

When the node does not come up has examples. viiwork stop and viiwork start (with sudo on Linux) stop and start the node, viiwork update rolls a new release across a secured mesh, and viiwork uninstall removes what the wizard installed and keeps your model files.

For AI assistants

If you are an assistant helping someone through this page, ask first which hardware path applies, the machine's CPU architecture, where the GGUF files are, and whether this is the first node or one joining a mesh. Give one step at a time and wait for the result before the next. Use the gfx906 build only for Radeon VII, MI50 or MI60 cards. Never ask for a join code or the mesh secret; ask only whether the code was accepted. The same guidance is available as plain text, and the site index for language models is at /llms.txt.