Enum ServingRefusal 

pub enum ServingRefusal {
    Step(DecodeRefusal),
    Undecoded {
        tensor: String,
        ty: GgmlType,
    },
    MissingTensor(String),
    NotResident {
        needed: f64,
        capacity: f64,
    },
    OnChipOverflow {
        needed: f64,
        capacity: f64,
    },
    BadDraft(String),
    Empty,
    Routed,
    NoHostLink,
    BadContexts(String),
    ContextBeyondModel {
        limit: u64,
        model: u64,
    },
    WideCacheOnArray {
        kv_bytes: f64,
    },
}

Why a serving could not be priced.

Variants

Step(DecodeRefusal)

Undecoded

A matrix in a format neither the array nor the vector units read.

Fields

tensor: String

ty: GgmlType

MissingTensor(String)

A tensor a stage names that the header does not hold.

NotResident

What the serving holds exceeds the card’s memory, and the host’s if it may use it.

Fields

needed: f64

capacity: f64

OnChipOverflow

The batch’s token-rows exceed the card’s on-chip memory.

Fields

needed: f64

capacity: f64

BadDraft(String)

A draft whose acceptance list is not one probability per drafted token.

Empty

A serving of no sequences, or no activation words.

Routed

A mixture of experts: which experts a pass reads depends on how its rows route, which the pricer does not model.

NoHostLink

A serving that may use host memory, on a card with no host link.

BadContexts(String)

A context mix whose shares do not sum to one, or a class past its own limit.

ContextBeyondModel

A context limit past the checkpoint’s own context length.

Fields

limit: u64

model: u64

WideCacheOnArray

Attention on the array over a cache whose elements are wider than its 8-bit codes.

Fields

kv_bytes: f64

Trait Implementations

impl Clone for ServingRefusal

fn clone(&self) -> ServingRefusal

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ServingRefusal

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<DecodeRefusal> for ServingRefusal

fn from(r: DecodeRefusal) -> ServingRefusal

Converts to this type from the input type.

impl PartialEq for ServingRefusal

fn eq(&self, other: &ServingRefusal) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for ServingRefusal

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for ServingRefusal