Module streamed 

A model as the RDU would run it: one lowered program per LAYER KIND, and a manifest.

Two layers whose specs are equal (crate::model::spec::Layer: the mixer, its rotation and window, the feed-forward block, the norms) are written by the author as the same program up to their names, so they lower to the same spatial program — measured field by field, the only differences are name labels. Qwen3.8’s 64-layer forward is not 64 programs; it is two (its Gated DeltaNet layers and its attention layers), Gemma 3’s are its windowed and its global attention, DeepSeek-V2’s its dense first layer and its expert layers: each applied layer after layer with a different set of weights bound each time, followed by the head. This module builds exactly that for any model the author writes: the kind programs, lowered and certified once each, and a manifest saying which kind runs at each layer and how that kind’s tensor names bind to that layer’s tensors.

Lowering cost is therefore independent of depth — one layer’s worth per kind, however many layers the model has — and so is the artifact a vendor ingests.

★ A WEIGHT MOVES AS IT IS STORED. The author declares every projection as a weight the deployment may store Q8_0; a product over one folds its int8 values and block scales on the chip (super::passes::from_q8) only where the checkpoint stores it so (Storage). A weight stored any other way is read dense, the host filling it through its binding.

Structs

LayerBinding

How one layer of the forward runs: which kind, and how the kind program’s tensor names map onto this layer’s tensors. Every tensor the kind program declares appears exactly once.

LayerKind

One lowered, certified layer program, standing for every layer of its kind.

StreamedModel

StreamedReject

A program of the bundle that did not lower: which one, and the whole-program front end’s own refusal — the statements it could not lift, by index and binder, or the fault that kept the composition from certifying.

Enums

Storage

Which of the weights the author declares Q8_0 the checkpoint stores Q8_0.

Functions

layer_kinds

The model’s layer kinds, in the order their first layer appears: each kind’s name and its layers. Two layers are of one kind when their authored programs (source_of: the text, and the weights it reads as stored Q8_0) are one program up to the names they give — the same shape_digest — so one lowered program serves both, bound to each layer’s own tensors by the renaming between their programs.

lower_batched_decode_streamed

Lower spec as an SN50 BATCHED decode deployment: batch sequences per step, each with its own token, position and carried state, every weight read once for all of them (the model’s Cut::Batch). The same three kinds of program as lower_decode_streamed — the embedding lookups, one program per layer kind, the head — over the stacked batch × d hidden states; the token ids toks and each sequence’s pos_s{b} are runtime inputs.

lower_batched_tp_decode_streamed

Lower spec as an SN50 BATCHED, TENSOR-PARALLEL decode deployment — batch sequences per step over tp chips, the serving shape: lower_batched_decode_streamed’s stacked programs with every layer kind lower_tp_decode_streamed’s program over the chips.

lower_decode_streamed

Lower spec as an SN50 DECODE deployment: the embedding lookup, one program per layer kind in which every attention layer appends to and attends over its KV cache of ctx positions, and a head that emits the next token. tok and pos are runtime inputs — the programs read them as data — so the bundle serves every step; the values the programs are lowered with (token 1, position 1) only seed their declarations.

lower_forward_streamed

Lower spec as one program per layer kind plus the head, and bind every layer to its kind.

lower_prefill_streamed

Lower spec’s PREFILL as an SN50 deployment: chunk tokens of one sequence per step, the cuts of the model’s Cut::Prefill — the chunk’s embedding lookups stacked, one program per layer kind taking the rows in order over the sequence’s single state (the decode step’s own, so a decode bundle continues from it), and a head reading the last row. Each row’s token and position are runtime inputs.

lower_tp_decode_streamed

Lower spec as an SN50 TENSOR-PARALLEL decode deployment over tp chips: the embedding lookup and the head on the replicated spine, and each layer kind one program over the chips — every chip its own heads and FFN columns, the partials summed by an all-reduce twice a layer, the chips’ slices of each op in one phase (see spatial::tensor_parallel). tp must be a width the model admits (fit::admits_tp); each kind program carries its manifest’s slices, so a host knows which rows or columns of which checkpoint tensor each chip loads.

rebind

The name layer to gives the tensor that layer from calls name. The hidden-state input is named by position (x for the first layer, xo{l-1} after); every per-layer tensor carries its layer’s index as a suffix (Wq0, x2n3); anything else — the shared RoPE tables (whose suffix is a rotation’s index, never a layer’s), the lowerings’ shared scalers — is the same tensor in every layer.

rebind_via

rebind for a bundle whose layers name their hidden-state input with input — the stacked X/bxo{l-1} of a batch. A per-sequence tensor keeps its sequence suffix (Kc3_s1 at layer 7 is Kc7_s1).

renaming

The renaming that turns program from into program to, bound name by bound name, when they share a shape_digest: the tokens aligned position by position, every word but a bound name kept exactly. None when they are not one program up to the names they bind.