Struct DecodeStep 

pub struct DecodeStep {
    pub deployment: Deployment,
    pub stages: Vec<Stage>,
    pub held: Vec<Held>,
}

What one token of a deployment reads, writes and computes, as the chain of its dependent stages.

Fields

deployment: Deployment

stages: Vec<Stage>

held: Vec<Held>

Bytes the deployment holds resident: every weight the step reads (the whole embedding table, not one row), the KV cache at the deployment’s context and the recurrent state, by how often a token moves them.

Implementations

impl DecodeStep

pub fn of(h: &GgufHeader, d: &Deployment) -> Result<DecodeStep, DecodeRefusal>

The step of d over the checkpoint h describes.

pub fn weight_bytes(&self) -> f64

pub fn state_read_bytes(&self) -> f64

pub fn state_write_bytes(&self) -> f64

pub fn ops(&self) -> f64

pub fn held_bytes(&self) -> f64

Every byte the deployment keeps resident.

Trait Implementations

impl Clone for DecodeStep

fn clone(&self) -> DecodeStep

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DecodeStep

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for DecodeStep

fn eq(&self, other: &DecodeStep) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for DecodeStep

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for DecodeStep