Struct EngineNumerics 

pub struct EngineNumerics {
    pub activations: Words,
    pub scaling: ActScaling,
    pub state: StateFormat,
    pub cache: CacheFormat,
    pub attention: AttentionMath,
    pub flush: StateFlush,
    pub fp_unit: FpUnit,
}

The knobs of the engine’s arithmetic.

Fields

activations: Words

scaling: ActScaling

How the activations a matrix reads are scaled: per 32 elements, or per the matrix format’s own block.

state: StateFormat

cache: CacheFormat

attention: AttentionMath

flush: StateFlush

How a Gated-DeltaNet memory is carried: rewritten every token, or held at its last flush with a log of the tokens since, read through the log and flushed once every tokens, or a window of tokens behind (Replay).

fp_unit: FpUnit

The multiplier every f32 product is computed by: the engine’s own, or the card’s floating-point DSP block.

Implementations

impl EngineNumerics

pub fn precise() -> EngineNumerics

The most precise knob point: two-word activations scaled per 32 elements, an f32 state, an f16 cache, attention on the vector units.

pub fn fastest() -> EngineNumerics

The fastest knob point the serving roofline prices: one-word activations scaled per weight block, an f16 state flushed every 16 tokens, a rotated int8 cache, attention on the array.

pub fn real_points() -> [EngineNumerics; 4]

The knob points held to the real checkpoints, both word counts at their most and least precise storage: precise; two words over an f16 state and an int8 cache on the array; one word over an f32 state and an f16 cache; fastest.

pub fn real_points_for(spec: &ModelSpec) -> [EngineNumerics; 4]

The knob points a real checkpoint of spec is held to: real_points when every attention layer norms its queries and keys, and otherwise the same points with the two-word point’s int8 cache rotated and the family’s fastest_for: un-normed keys carry outlier channels an unrotated int8 cache loses and the rotation spreads. Measured on Llama-3.2-1B against llama.cpp’s runs of its corpus: two words on an int8 cache followed the f32 run 458 steps to the shipped run’s 462, rotated 501.

pub fn fastest_for(spec: &ModelSpec) -> EngineNumerics

The fastest knob point spec’s family is held to: fastest, its attention on the vector units when a layer’s keys are not normed (Llama-3.2-1B’s logits after the prompt sat 0.098 from llama.cpp’s f32 run’s on the vector units, within the shipped run’s 0.080 by two standard errors, and 0.112 on the array).

pub fn all() -> Vec<EngineNumerics>

Every knob point the engine runs.

Trait Implementations

impl Clone for EngineNumerics

fn clone(&self) -> EngineNumerics

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EngineNumerics

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for EngineNumerics

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for EngineNumerics

fn eq(&self, other: &EngineNumerics) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for EngineNumerics

impl Eq for EngineNumerics

impl StructuralPartialEq for EngineNumerics