Core concepts
Clystria turns a fleet of unlike machines into one computer. Five ideas do the heavy lifting.
Nodes & the agent
A node is any machine running the Clystria agent. On startup the agent profiles the hardware — GPU model, VRAM, PCIe/NVLink topology, memory bandwidth and thermal headroom — and advertises it to the pool. Nodes gossip over the LAN; there is no central master to lose.
Because the agent measures real throughput rather than trusting spec sheets, a thermally-throttled 4090 is weighted differently from a cool one. The pool always reflects reality.
The unified pool
Every node's VRAM is presented as a single addressable space. A model larger than any single GPU can hold is simply split across the pool — its layers stream between cards over the fastest available interconnect. Add a machine and the pool grows the instant it powers on.
Layer placement
When you deploy a model, the scheduler solves a bin-packing problem: fit the layers, the KV-cache, and the activation buffers onto the available GPUs while minimizing cross-node traffic. It accounts for:
- VRAM capacity and current occupancy per GPU
- Interconnect bandwidth between candidate nodes
- Measured tokens/sec of each device under load
- Tensor- vs pipeline-parallel boundaries for the architecture
You never write a shard map. If a better placement exists, the control loop migrates layers to it during low traffic.
Routing & balancing
Incoming requests hit a router that tracks per-replica queue depth and KV-cache pressure, then dispatches to the replica that will finish first. Long-context and short-context requests are balanced separately so a 100k-token prompt never head-of-lines a chat turn.
Self-healing
The control loop heartbeats every node. When one drops — a yanked cable, a crash, a reboot — Clystria:
- Drains in-flight requests off the dead replica's router slot.
- Recomputes placement for the orphaned layers across survivors.
- Re-forges those layers and resumes serving, typically before the token stream stutters.
When the node returns, it rejoins and the pool rebalances back. No tickets, no pager, no human in the loop.
Security & data boundary
Nodes authenticate to a cluster with a rotating join token, and all inter-node traffic is mutually authenticated and encrypted. Model weights and every token of every request stay inside your network — Clystria has no cloud dependency and phones nothing home. Your intelligence never leaves the building.