Enum KvStorage 

pub enum KvStorage {
    Exact,
    F16,
}

How an attention block’s cache keeps the rows it is given.

Variants

Exact

As computed, in f64.

F16

Each element rounded to f32 and then to binary16, as llama.cpp’s default f16 cache stores the f32 rows it computes; every read of the cache takes the rounded row.

Trait Implementations

impl Clone for KvStorage

fn clone(&self) -> KvStorage

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for KvStorage

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for KvStorage

fn default() -> KvStorage

Returns the “default value” for a type.

impl PartialEq for KvStorage

fn eq(&self, other: &KvStorage) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for KvStorage

impl Eq for KvStorage

impl StructuralPartialEq for KvStorage