Enum AttnForm 

pub enum AttnForm {
    Current,
    Cached {
        ctx: usize,
    },
}

How an attention layer reads its keys and values.

Variants

Current

The current token’s own key and value — the forward of one token with no history.

Cached

A decode step’s KV cache of ctx positions: this token’s key and value are appended as row pos, and the layer attends over the first pos rows. The caches are the step’s carried state.

Fields

ctx: usize

Trait Implementations

impl Clone for AttnForm

fn clone(&self) -> AttnForm

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AttnForm

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for AttnForm

fn eq(&self, other: &AttnForm) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for AttnForm

impl Eq for AttnForm

impl StructuralPartialEq for AttnForm