Skip to main content

Module capability

Module capability 

Source
Expand description

What the decode engine runs: a checkpoint held, feature by feature and tensor by tensor, to the program super::program::lower reads off the model’s one program and the units compute, and every gap refused by name.

The lowering reads a ModelSpec and writes the commands of a Qwen3.5/3.8-shaped step: RMSNorm before each block, a Gated-DeltaNet or grouped-query attention mixer, a dense SwiGLU feed-forward block, the residual adds, the final norm and the head. A model with anything else would lower to a program that computes something other than the model, and nothing downstream would notice: the tensors of a GELU block are the tensors of a SwiGLU block. So before a checkpoint becomes a program, refusals names each feature the engine does not carry (spec_refusals), and then each tensor the program reads that the checkpoint lacks or stores in a format no unit reads: the array reads numerics::reads’s block formats, the vector unit plain floats.

An empty list is the engine’s claim that it computes the model; admit is that claim as a gate.

A checkpoint the engine computes runs on an engine build when the build has every unit, width and format its step needs (needs): covers holds them against the build’s EngineParams, the dual of EngineParams::union (a build covers a step exactly when the union of the two is the build), and names every gap. runs_on is both at once: whether a checkpoint’s header runs on a build, in the time it takes to read the header.

Structs§

Needs
What a checkpoint’s step needs of an engine build: the engine its jobs need (EngineParams::of) and the block formats of the matrices a hybrid build streams, in their ggml ids’ order.
Row
One checkpoint’s row of the capability matrix.

Enums§

Gap
One thing a step needs that an engine build does not have.
Refusal
One thing a checkpoint needs that the engine does not do.
Verdict
The engine’s verdict on a checkpoint described by its header.

Functions§

admit
The engine’s claim that it computes the checkpoint, or the gaps that stand in the way.
covers
Whether the engine build runs the step step needs, or every gap: each width it holds at least as many of, each block format its matrix unit reads in rows at least as long (the vector side’s registers among them, as deep as the step’s deepest job that cannot run a chunk at a time), the recurrence unit of the same heads, the attention unit of the same heads over at least as many positions, and on a hybrid build every streamed format decoded. A build covers a step exactly when the union of the two (EngineParams::union) is the build. The vector unit’s function units set how fast a function row runs, not what runs, so they leave no gap. Every field of the step’s engine is named here, so a field EngineParams gains does not compile until it is judged.
judge
The verdict on the checkpoint header describes: its architecture read as a ModelSpec, its tensor directory as the formats it stores. A tensor of a type id no reader knows is refused as such.
matrix
The verdict on every header ledger (*.json) in dir, by file name.
needs
What the step of the checkpoint header describes needs, at the engine’s fastest point, one sequence over at most ctx positions.
refusals
Every gap between the checkpoint and the engine: spec_refusals, and when there are none, the frequency factors a rotation scaled by them turns by (rope_freqs.weight, written into its rotary tables) when tensors (the checkpoint’s directory, name and format) lacks them, and each tensor the program reads that tensors lacks or stores in a format no unit reads.
runs_on
Whether the checkpoint header describes runs on the engine build at ctx positions: the engine computes it (judge) and the build covers its step (covers). Otherwise every reason, a sentence each.
spec_refusals
Each feature of spec the lowering does not carry, layer by layer, then the model’s own.
summarize
Each distinct gap once, in the order first met, with the layers it stands in: every layer when it is all of a model’s layers.