Struct AttentionSpec 

pub struct AttentionSpec {

    pub q: Symbol,
    pub kc: Symbol,
    pub vc: Symbol,
    pub out: Symbol,
    pub h: usize,
    pub kv: usize,
    pub hd: usize,
    pub vd: usize,
    pub scale: Option<Factor>,
    pub softcap: Option<Factor>,
    pub l: usize,
    pub pos: Option<RowIndex>,
    pub rows: usize,
    pub window: Option<Window>,
    pub sinks: Option<Symbol>,
}

ctx = attention(q over kc, vc) — single-query multi-head GQA attention: q is 1 × (h·hd), kc is l × (kv·hd) and vc l × (kv·vd) (the cached keys/values), query head g attends to kv head g / (h/kv) and writes vd values. pos is the position bound of the masked up to P form (the KV cache as carried state); None is the unmasked form that attends to every cached row.

Fields

q: Symbol

kc: Symbol

vc: Symbol

out: Symbol

h: usize

kv: usize

hd: usize

vd: usize

The value head width: hd, or narrower (V holding kv·vd columns).

scale: Option<Factor>

The score scale a program gives (scaled by S); None is the default 1/√hd, which a given scale equal to it also lifts as.

softcap: Option<Factor>

The capped at C form’s C: every scaled score s taken as C·tanh(s/C) before the softmax (Gemma 2).

l: usize

pos: Option<RowIndex>

The masked up to P form’s bound: attend over the first P of the l cached positions. For a block of rows query rows (causal attention … from P) it is the FIRST row’s: row r (0-based) attends P + r positions.

rows: usize

Query rows: 1 for the single-query form, T for a causal block.

window: Option<Window>

The positions a query keeps of the n it attends: the last W (within a window of W), or those of its own chunk of C (within chunks of C).

sinks: Option<Symbol>

The with sinks S form’s S: a logit per query head joining its softmax, taking no value.

Trait Implementations

impl Clone for AttentionSpec

fn clone(&self) -> AttentionSpec

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AttentionSpec

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for AttentionSpec

fn eq(&self, other: &AttentionSpec) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for AttentionSpec

impl StructuralPartialEq for AttentionSpec