Enum BlockState 

pub enum BlockState {
    Delta {
        ring: u64,
        channels: usize,
        taps: usize,
        memory: u64,
        log: Option<u64>,
        entry: u64,
        k_heads: usize,
        v_heads: usize,
        width: usize,
    },
    Attention {
        keys: u64,
        values: u64,
        key_scales: Option<u64>,
        value_scales: Option<u64>,
        kv_heads: usize,
        width: usize,
        value_width: usize,
    },
}

Where one block keeps a sequence’s state, from the start of the sequence’s region.

Variants

Delta

Fields

ring: u64

The conv ring: taps − 1 rows of channels elements, position p’s row at p mod (taps − 1).

channels: usize

taps: usize

memory: u64

The memory: per value head width rows of width elements.

log: Option<u64>

A lazily flushed memory’s log: the count of tokens it holds and the slot of its oldest (four bytes each), then a line on, Replay::capacity slots of entry bytes, the log’s ith token in slot (oldest + i) mod capacity, each every key group’s key, every value head’s values, every value head’s decay exponent and every value head’s gate, each field from a whole beat of sixteen elements (LogFields).

entry: u64

k_heads: usize

v_heads: usize

width: usize

Attention

Fields

keys: u64

Per position, every cached head’s key row; per position, every cached head’s value row: binary16 elements, or int8 codes.

values: u64

key_scales: Option<u64>

An int8 cache’s scales, per position every cached head’s key row’s, then its value row’s, one f16 each.

value_scales: Option<u64>

kv_heads: usize

width: usize

value_width: usize

Trait Implementations

impl Clone for BlockState

fn clone(&self) -> BlockState

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BlockState

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for BlockState

fn eq(&self, other: &BlockState) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for BlockState

impl StructuralPartialEq for BlockState