S2
Nerv's inference engine: written from scratch, shipped in the image, specialised for the GB10 rather than portable across accelerators, and running behind axon.
The shape
Two build-time artifacts and two run-time policies, against one cost model. Everything decidable from the model and the silicon is decided once, ahead of time, and recorded. Everything that depends on system state is decided at run time by exactly two policies.
| the file | One self-contained artifact per model: weights in access-frequency order, a graph over engine primitives, precision per tensor, tokenizer, template, drafter, ledger, provenance. No runtime options. |
| the graphs | An ahead-of-time compilation of file and hardware into replayable CUDA graphs and an arena layout. The runtime replays them. |
| residency | What occupies which memory tier right now, as one ranked table rather than five mechanisms. |
| depth | How far to speculate on this request, given measured acceptance and the deadline. |
| the ledger | One table of measured rates and per-model byte costs. Every decision above is a query against it. |
One unit: bytes read per accepted token
Residency is cut by reconstructibility, not by size and not by file-versus-driver.
Cache is any byte that exists somewhere else: an expert extent rebuilt in 3–6 ms from NVMe, a model's hot block at about 5 s cold, a device-tagged copy that is a memcpy from page cache, a vision tower, an entire small model. Dropping cache is always safe; only the price differs.
State is any byte that exists nowhere else: live KV, the recurrent state of a hybrid model, sampler state. It cannot be dropped, only ended or demoted at a re-prefill cost measured in seconds.
Once that cut is made, models are not the unit of residency — bytes are, ranked by value density. A model switch is a priced transaction rather than a mode, and "several models loaded at once" is not a meaningful claim under this scheme.
Full design: docs/s2/