Expand description
What the decode engine runs: a checkpoint held, feature by feature and tensor by tensor, to the program
super::program::lower reads off the model’s one program and the units compute, and every gap refused by name.
The lowering reads a ModelSpec and writes the commands of a Qwen3.5/3.8-shaped step: RMSNorm before each
block, a Gated-DeltaNet or grouped-query attention mixer, a dense SwiGLU feed-forward block, the residual adds, the
final norm and the head. A model with anything else would lower to a program that computes something other than the
model, and nothing downstream would notice: the tensors of a GELU block are the tensors of a SwiGLU block. So
before a checkpoint becomes a program, refusals names each feature the engine does not carry
(spec_refusals), and then each tensor the program reads that the checkpoint lacks or stores in a format no unit
reads: the array reads numerics::reads’s block formats, the vector unit plain floats.
An empty list is the engine’s claim that it computes the model; admit is that claim as a gate.
A checkpoint the engine computes runs on an engine build when the build has every unit, width and format its step
needs (needs): covers holds them against the build’s EngineParams, the dual of EngineParams::union
(a build covers a step exactly when the union of the two is the build), and names every gap. runs_on is both
at once: whether a checkpoint’s header runs on a build, in the time it takes to read the header.
Structs§
- Needs
- What a checkpoint’s step needs of an engine build: the engine its jobs need (
EngineParams::of) and the block formats of the matrices a hybrid build streams, in their ggml ids’ order. - Row
- One checkpoint’s row of the capability matrix.
Enums§
- Gap
- One thing a step needs that an engine build does not have.
- Refusal
- One thing a checkpoint needs that the engine does not do.
- Verdict
- The engine’s verdict on a checkpoint described by its header.
Functions§
- admit
- The engine’s claim that it computes the checkpoint, or the gaps that stand in the way.
- covers
- Whether the engine
buildruns the stepstepneeds, or every gap: each width it holds at least as many of, each block format its matrix unit reads in rows at least as long (the vector side’s registers among them, as deep as the step’s deepest job that cannot run a chunk at a time), the recurrence unit of the same heads, the attention unit of the same heads over at least as many positions, and on a hybrid build every streamed format decoded. A build covers a step exactly when the union of the two (EngineParams::union) is the build. The vector unit’s function units set how fast a function row runs, not what runs, so they leave no gap. Every field of the step’s engine is named here, so a fieldEngineParamsgains does not compile until it is judged. - judge
- The verdict on the checkpoint
headerdescribes: its architecture read as aModelSpec, its tensor directory as the formats it stores. A tensor of a type id no reader knows is refused as such. - matrix
- The verdict on every header ledger (
*.json) indir, by file name. - needs
- What the step of the checkpoint
headerdescribes needs, at the engine’s fastest point, one sequence over at mostctxpositions. - refusals
- Every gap between the checkpoint and the engine:
spec_refusals, and when there are none, the frequency factors a rotation scaled by them turns by (rope_freqs.weight, written into its rotary tables) whentensors(the checkpoint’s directory, name and format) lacks them, and each tensor the program reads thattensorslacks or stores in a format no unit reads. - runs_on
- Whether the checkpoint
headerdescribes runs on the enginebuildatctxpositions: the engine computes it (judge) and the build covers its step (covers). Otherwise every reason, a sentence each. - spec_
refusals - Each feature of
specthe lowering does not carry, layer by layer, then the model’s own. - summarize
- Each distinct gap once, in the order first met, with the layers it stands in:
every layerwhen it is all of a model’slayers.