Module twin 

The decode engine’s cycle twin: a step’s program priced command by command on the engine’s units.

Each command asks its units for work (Demand): the stream engines for its weights from the card’s memory and for the state and caches its rows’ sequences read and write, each way of moving them at the share of the memory’s peak it sustains (crate::spatial::serving::Streams); the array and the block-scale lanes for a block-format matrix; the vector units for the norms, the attention, the recurrence, a float matrix and the elementwise steps between them. A command takes as long as its slowest unit, the others overlapped with it. Weights stream ahead of the command that reads them into an on-chip window while earlier commands run (schedule): a command’s own state and cache traffic is served first, and the memory time it leaves spare fetches the next commands’ weights as far as the window holds.

The weights, the traffic of each kind, the array’s multiply-adds and the lanes’ cycles are the pricer’s (crate::spatial::serving::PassShape) term for term; the twin’s vector work is the engine’s own, every head of a per-head norm and every elementwise step the pricer leaves out.

Structs

Demand

The work one command asks of each unit.

Predicted

The twin’s prediction of the 27B on the D5005’s standard plan: every knob point the real checkpoints are held to, at each of PREDICTED_BATCHES the card holds and at the largest batch it holds, every sequence at PREDICTED_POSITION.

PredictedRow

One configuration the twin predicts, beside the roofline’s price of it.

SequenceRows

One sequence a step advances: the rows it brings and the positions its caches hold before them.

Span

One command on the step’s timeline.

Step

The rows a step runs: every sequence it advances, and whether the rows past each sequence’s first are drafts the step verifies, so every candidate prefix’s state is kept.

Timeline

A step on the card: every command’s span and the step’s length.

Enums

Bound

Which unit a command waited on.

TwinRefusal

Why the twin cannot price a program.

Constants

EXP_OPS

f32 operations an element of each of the vector unit’s functions takes, as super::numerics sequences them: exp its range reduction, its cubic and its scaling; recip and rsqrt a seed and two Newton steps; sigmoid an exp, an add and a recip; silu a sigmoid and a product; softplus an exp and an ln1p.

PREDICTED_BATCHES

Batches the prediction prices where the card holds them, every sequence at PREDICTED_POSITION; with them each knob point’s largest batch the card holds (crate::spatial::serving::max_batch).

PREDICTED_HEADER

The header ledger the prediction prices.

PREDICTED_LIMIT

Each sequence’s context limit: the roofline’s 2k-of-4k class, its cache paged.

PREDICTED_POSITION

Positions each sequence’s caches hold before the step.

RECIP_OPS

RSQRT_OPS

SIGMOID_OPS

SILU_OPS

SOFTPLUS_OPS

Functions

demands

The work of every command of program over step on the checkpoint h describes.

predicted

The prediction, from the header ledgers under decode_dir (benchmarks/silicon/decode): the architecture the ledger describes (GgufHeader::spec), the program of each knob point priced and scheduled.

schedule

The step’s commands in order on card: each starts when the one before it ends and lasts as long as its slowest unit; the memory serves the command’s own traffic, then its weights not yet fetched, then the following commands’ weights in order while window bytes hold them.

serving_of

The serving the pricer describes for a step of batch sequences at n, each attending to attended positions.

steady_steps

The steps a steady decode of batch sequences a row each at position cycles through: one, or under a lazy flush one for every phase of the flush window, sequence i of phase p having logged (i + p) mod tokens tokens past the window a trailing flush is replaying — so their mean is a mean step.