Enum AttentionMath 

pub enum AttentionMath {
    Vector,
    Array,
}

Where attention’s two products over the cache run.

Variants

Vector

Scores and the weighted sum in f32 on the vector units, over the cache as stored.

Array

Scores and the weighted sum as integer dot products on the array: int8 queries against an int8 cache, and 8-bit probabilities against it, two passes each (an 8-bit cache code is two digits). The softmax stays on the vector units.

Trait Implementations

impl Clone for AttentionMath

fn clone(&self) -> AttentionMath

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AttentionMath

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for AttentionMath

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for AttentionMath

fn eq(&self, other: &AttentionMath) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for AttentionMath

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Copy for AttentionMath

impl Eq for AttentionMath

impl StructuralPartialEq for AttentionMath