Module reference 

The reference forward: what a ModelSpec computes, evaluated in f64 on the CPU over the checkpoint’s own weights.

This is the oracle every compiled lane and every competitor is measured against — the “same-weights” reference: the weights are exactly the checkpoint’s (each dequantized as ggml defines its format), and everything after the weights is f64, so the only error left in a lane’s logits is the lane’s own. It runs one prompt as a batch per layer (each weight is read once per pass, not once per token) and then decodes with a KV cache.

The operations follow llama.cpp’s graph for each architecture (src/models/<arch>.cpp at the pinned commit).

Structs

Reference

The reference model over a checkpoint.

RoundingCall

A rounding the reference made within NEAR_ROUNDING of a half-way point: value element of the activation whose f32 values hash to activation (word 0 the int8 or two-pass high word, 1 the low word, CACHE_WORD a cached row’s binary16), first made in the step ending at position, margin levels from the half-way point (a cached value’s in levels of its row’s scale, its largest magnitude over 127). An activation is one decision however many projections read it — the lane quantizes it once — so a call names the activation, not the read.

RoutingCall

The closest call a router made: at position (0-based) in layer layer, the last expert (or expert group) it chose scored margin above the first it passed over, on a selection score of score. A run in another precision may choose otherwise wherever margin is below what that precision resolves at score.

Tensors

Tensors held in memory, each in a GGUF format.

Enums

Activations

How the reference’s projections by quantized weights read their activations. Exact is the model’s arithmetic; the others are the activations a quantized matmul reads under llama.cpp’s rule — a quantized weight reads a quantized activation — computed as the GPU lane’s kernels compute them, so a lane can be held to its own precision.

KvStorage

How an attention block’s cache keeps the rows it is given.

NearCall

A decision the reference made near its other outcome — one a run in another precision may make the other way.

ReferenceError

Why the reference cannot run a checkpoint.

StateStorage

How a gated-delta layer keeps its recurrent memory between steps.

Constants

CACHE_WORD

The word a cached value’s binary16 rounding is recorded under (RoundingCall::word).

DEFAULT_CACHE_BUDGET

The bytes the reference keeps of a checkpoint’s tensors by default: every tensor of a model up to this size is read from the file once.

NEAR_ROUNDING

How near a half-way point between two levels a rounding the reference records lies, in levels: within it, a run whose activation differs from the reference’s by its own f32 rounding may take the other level.

NEAR_ROUTING

How near a tie, as a fraction of the chosen score, a routing call the reference records lies.

Traits

WeightSource

Where the reference reads a model’s weights: a checkpoint, or tensors held in memory (a test’s tiny model). Tensors are named and laid out as GGUF names and lays them out (dims[0] the fastest).

Functions

argmax

The index of the largest logit (the first of equals).

quantize_activations

x as a projection by a quantized weight reads it under mode, a block of 32 values at a time.