Module machine 

The decode engine’s machine: a step as the jobs its sequencer runs over the card’s memory, every byte where it lives.

super::program says what a step computes; the machine says where each number is kept and how the units move it. The card’s memory (Layout) holds the weight image — every tensor’s GGUF bytes in the order the program first reads them — the rotary tables, and a region for every sequence the machine holds: per Gated-DeltaNet block its conv ring, its memory and its log, per attention block its key and value caches, each in the format the knobs name. A step’s rows are its row table: each row’s sequence, position and token.

lower turns each command into one Job. The vector unit’s jobs (VectorJob) name what each pass loads into the vector registers — a span of a token-row’s slot, a tensor’s elements, the row’s rotary table, its sequence’s ring — the micro-program it runs and what it stores, every pass a row or a head of a row. The array’s, the recurrence unit’s and the attention unit’s jobs name their tensors, slots and blocks. Machine::step runs the jobs over the memory, reading every value as stored and writing it as stored, and computes what super::model::EngineModel::step computes, bit for bit: the jobs are what the sequencer issues, and the memory is what the card holds.

Structs

DeltaNorm

A recurrence job’s read-outs gated and normed where they are made (fold_delta_norms): each head RMS-normed by tensor weight and gated by silu of the head’s elements of z, eps the norm’s epsilon — the gated norm (Command::GatedNorm’s job) the read-outs took next, run as part of the job.

Layout

The card’s memory as a machine lays it out.

Load

LogFields

Where a log entry keeps its fields, in elements: every key group’s key, every value head’s values, every value head’s decay exponent, every value head’s gate, each field from a whole beat.

Machine

A checkpoint on the machine: its layout, its jobs, and the card’s memory they run over.

Placed

A tensor of the weight image: its format, its dims, where its bytes start and how many there are.

Put

Row

One row of a step’s row table: its sequence, its position and token, and where its position’s row sits in its sequence’s conv rings, position mod (taps − 1), the host’s division so the engine never divides.

StoreLayout

The token-row store as the engine holds it: beats of LANES f32 elements, slot s’s row r from beat base + r·beats of its entry, every slot wide enough for the widest vector the step’s jobs write to it, and two slots over the same beats only when no point of the step holds both. A greedy store (StoreLayout::greedy) keeps each row’s logits as their arg-max.

Trace

The machine before one of a step’s jobs runs: the job’s index (the job count once the last has run), the step’s rows, the store and the card’s memory.

VectorJob

A job of the vector unit: for every row of the step, in order, and every head of the row, a pass that loads its registers, runs program and stores from them.

Enums

BlockState

Where one block keeps a sequence’s state, from the start of the sequence’s region.

CachePart

Which part of a cache row a store writes.

Fold

How a matrix job’s sums meet the elements of its output slot (fold): each written as it is, added to the element it lands on (a residual connection, Command::Add’s), or multiplied by the SiLU of the element, the gate’s sum a gated FFN’s product takes (Command::SwiGlu’s, the element its gate).

Job

One job of the sequencer.

Order

The order a vector job’s passes run in.

Reg

A register a store reads.

Sink

Where a vector job’s store writes, for the pass’s row and head.

Source

Where a vector job’s load reads, for the pass’s row and head.

Constants

LANES

Elements a beat carries: the vector unit’s lanes.

LINE

Bytes every placed thing starts on: one line of the card’s memory.

SREGS

STORE_ORDERS

The orders StoreLayout::of tries past the widest first, each the one before shuffled, before it takes the shortest row it laid.

VREGS

Vector and scalar registers a vector job’s passes run over.

Functions

cache_row_bytes

Bytes of one head row of width elements in a cache of format: binary16 elements, or int8 codes.

decode

The jobs words stand for, or where a word stands for none.

encode

A job as the sequencer reads it: 64-bit words, a header and then the job’s own.

fold

The step jobs with each matrix whose sums only a residual add or a gated FFN’s product takes folding them into that job’s output slot (Fold): a matrix job and the add of its output into another slot become one job adding its sums into that slot; a gate’s and an up projection’s matrix jobs and their product become the gate’s sums into the product’s slot and the up projection’s folded into them through the gate’s SiLU. Jobs fold only where they stand next to each other and the slots the step no longer writes are read by no job after (StoreLayout::live_out), so every value the step computes is the one it computed, and those slots leave the store.

fold_delta_norms

The step jobs with each recurrence job whose read-outs a gated norm in place takes next carrying that norm (DeltaNorm): the two jobs become one, the recurrence’s read-outs gated and normed where they are made. The norm’s gate is a slot other than the read-outs’, so it is still the one the norm read. The machine runs a job with a norm as the recurrence then the norm’s own job, so every value the step computes is the one it computed.

gated_norm_job

Command::Add’s job: from added into to, n elements. Command::GatedNorm’s job: each head of width elements of from (n a row) RMS-normed by tensor weight and gated by silu of the head’s elements of z, into to, eps the norm’s epsilon.

lower

The jobs of program over the tensors layout places, eps the norms’ epsilon: one a command, its unit’s, and for a rotated cache’s attention a second, the vector unit’s rotation back.

state_bytes

Bytes of one element of a recurrent state stored as format.

Type Aliases

Store

The token-row store: each slot’s vector for every row of the step, in the step’s row order.