MatterWave on a Raspberry Pi

One small ARM box runs the whole product for a household or a small team: the API serving the web app, Postgres, and a CPU llama-server decoding grt-nano. No cloud is in the loop. When the box has a network it is designed to mirror its conversations to your account on the platform; when it does not, it keeps working.

Hardware

• Raspberry Pi 5, 8 GB, the recommended box. A Pi 4 with 4 GB runs grt-nano only.
• An SSD over USB 3 or NVMe for the OS and Postgres. An SD card wears out under a database.
• Wired Ethernet or Wi-Fi on the same LAN as the people using it.
• A 64-bit OS with podman 5.0+ for the quadlets: Raspberry Pi OS on Debian trixie. Ubuntu 24.04 (podman 4.9) and Raspberry Pi OS bookworm (4.3) cannot run these units; there, run the compose file instead (--compose, docker compose or podman-compose).

What runs

Three containers, as podman quadlets (systemd user units) or one compose file, no Kubernetes:

Unit	Image	Role
postgres	postgres:16-bookworm (multi-arch)	the database, data on the SSD
api	the MatterWave image, built for linux/arm64	API and web app, on 127.0.0.1:MW_EDGE_PORT (8080) plus the one LAN address MW_EDGE_BIND names
decode	the GRT llama.cpp fork, built for aarch64	serves grt-nano on http://decode:8000/v1

The API talks to llama-server directly; there is no LiteLLM gateway. Sign-in is local (MW_AUTH_MODE=none): everyone on the LAN is the one local user, so keep the box off the open internet.

grt-nano has a 1024-token window. Compaction is what lets a thread on it run past two or three turns, and it runs on the same CPU.

What does not run

Expected to be reported as off by GET /v1 and /api/auth-config (not yet checked on a box):

• Billing, plans, Stripe: there is no org to bill.
• Moderation: no provider key, so every prompt passes. Acceptable for a private box.
• Web search: needs a network and a key.
• Attachment scanning: ClamAV needs about 2 GB of RAM on its own.
• Vision and embeddings: no model on the box supports them.
• Zitadel, SSO, members, invites, the admin console: features of a platform org, not a box.
• qwen3-8b, vLLM, GPU decode: no hardware for them.
• Mail, error tracking, tracing and Redis are optional and off.

The tree

deploy/edge/ holds:

• quadlet/, the units for postgres, api and decode, the matterwave network and the matterwave-pg / matterwave-blobs volumes; docker-compose.yml, the same for compose;

• edge.env.example, the box's settings, copied to ~/.config/matterwave/edge.env. No secrets: the Postgres password and session secret are generated into a 0600 secrets.env beside it.

  MW_ENV=edge
  MW_AUTH_MODE=none
  MW_UPSTREAM_BASE_URL=http://decode:8000/v1
  MW_LITELLM_VIRTUAL_KEYS=false
  MW_DEFAULT_MODEL=grt-nano
  MW_MODEL_CONTEXT_WINDOWS=grt-nano=1024
  MW_BLOB_BACKEND=fs
  MW_EDGE_BIND=127.0.0.1          # the Pi's LAN address to let the house in; never 0.0.0.0
  MW_EDGE_PORT=8080               # the one place the port is set; the units and scripts read it
  MW_EDGE_DECODE_CTX=1024         # llama-server's -c, equal to the window above
  MW_EDGE_SYNC_URL=               # the platform, once paired (below)
  MW_EDGE_DEVICE_ID=              # this box's id, minted at pairing

• first-boot.sh: writes the settings and secrets, checks the GGUF's sha256, pulls and tags the images, installs the units, starts Postgres and runs the migrations. It seeds nothing;

• edge-up.sh, which starts the units and proves the box works.

Operating details (binding, backup, measuring memory) are in deploy/edge/README.md.

First boot

1. Install a 64-bit OS on the SSD, boot, and install podman (or docker).
2. Copy the repo's deploy/edge/ to the box, put the GGUF at ~/.local/share/matterwave/models/grt-spark.gguf, fill in the registry and image tags in ~/.config/matterwave/edge.env, and run deploy/edge/first-boot.sh (add --compose without quadlets).
3. Run deploy/edge/edge-up.sh. It starts the units and checks, in order: /readyz says ok, /api/models lists grt-nano with its 1024-token window, and one streamed turn completes. It stops at the first check that fails and says which.
4. Open http://<pi-address>:<MW_EDGE_PORT> (8080 by default) from another machine on the LAN, once MW_EDGE_BIND is the Pi's LAN address (loopback only otherwise).

Pairing and sync

• Pairing. On the platform, an org owner or admin opens Settings → Devices, names the box and makes a pairing code, shown once and valid once for ten minutes. On the Pi, Settings → Devices takes the platform's address and the code (or deploy/edge/edge-up.sh --check --pair <url> <code> from its shell). The Pi gets an API key with the single scope conversations:import, named Edge: <box name>. A leaked Pi key cannot chat as you or read your platform conversations: the scope lets it add conversations to your account and nothing else. Unpair from either side; unpairing on the platform revokes the key and the Pi stops.
• Where it is kept. The key, the platform's address and the Pi's device id are in pairing.json (mode 0600) on the matterwave-edge-state volume, beside the sync cursor and status (MW_EDGE_SYNC_STATE). A Pi paired by hand instead sets MW_EDGE_SYNC_URL and MW_EDGE_DEVICE_ID in edge.env and MW_EDGE_SYNC_KEY in secrets.env; a pairing file wins.
• Sync is one way, Pi to platform, run by the worker unit (the API image's worker, without Redis) every minute. It pushes conversations with a message newer than the last acknowledged one to POST /api/sync/conversations, one request per conversation, in the shape of /api/me/export plus an origin on each message. Ids are made on the Pi, so a re-upload is a no-op. Mirrored conversations record the device and model on each message and are not billed. The platform never writes back to the Pi.
• Attachments are not mirrored: the message keeps the part that names the file (and a document's extracted text), the bytes stay on the Pi. Platform uploads pass a virus scan the Pi does not run; mirroring files is a separate, later step.
• Offline is a state, not an error: the job backs off 1, 2, 4 ... 30 minutes and resets on the first success. A revoked key (401/403) or a platform that no longer reads the Pi's format_version (422) stops the job until the Pi is paired again or updated.
• Refused messages do not stop it: a message the platform refuses for good (a 4xx other than 429) is skipped and counted, and everything after it still goes. Edits to a message the Pi already sent (a tool-using answer finishing) are sent again and replace the platform copy; a conversation deleted on the platform is never put back, even after the platform purges it.
• Status. The Pi's header shows Synced 4 minutes ago, Offline, last synced 2 hours ago, Syncing 12 conversations…, Synced 4 minutes ago, 2 messages not mirrored, Sync key rejected: pair again or Box out of date for this platform, with the number of messages waiting and the last error on hover; nothing when the Pi is not paired. It reads GET /api/sync/status on the Pi, so it renders offline.

Update

Put the new tags (MW_EDGE_API_TAG, MW_EDGE_DECODE_TAG) in edge.env, back up, then:

deploy/edge/first-boot.sh --pull    # pull, retag, run the migrations
deploy/edge/edge-up.sh              # restart api and decode on the new images, prove it

CI builds the API image for arm64 (edge-image.yml, under QEMU, not yet proven green) and pushes it as matterwave-api:<sha12>-arm64, with the manifest list at the bare <sha12>. No arm64 build has been run from this tree, by CI or by hand (deploy/edge/build-arm64.md). The decode image is not built: whether the GRT fork's kernels build and agree on aarch64 is not yet established; it has only been built for x86_64.

Back up

Postgres holds everything except uploaded files, which are in the matterwave-blobs volume.

podman exec matterwave-postgres pg_dump -U matterwave -d matterwave -Fc > mw-$(date +%F).dump
podman volume export matterwave-blobs > mw-blobs-$(date +%F).tar

Keep both, and ~/.config/matterwave/secrets.env, off the box and encrypted. Restore is pg_restore into the running matterwave-postgres after first-boot.sh, then edge-up.sh.

Memory

Estimated from the model profiles' requests and limits, not measured on a Pi:

Box	OS + Postgres + API	Decode	Fits
Pi 4, 4 GB	~1.2 GB	grt-nano ~0.3 GB	grt-nano only
Pi 5, 8 GB	~1.2 GB	grt-nano + qwen3-0.6b at 8k context ~4 GB	two models
Pi 5, 16 GB	~1.2 GB	qwen3-0.6b + a 1.7B-class model	qwen3-8b does not

Run llama-server with -t 4 on every Pi: no more threads than cores.