Struct ServingPlan 

pub struct ServingPlan {

    pub tensor_parallel: usize,
    pub data_parallel: usize,
    pub ctx: usize,
    pub hbm_bytes_per_chip: usize,
    pub weights_per_chip_bytes: usize,
    pub sequence_bytes_per_chip: usize,
    pub max_sequences_per_replica: usize,
    pub max_sequences: usize,
    pub sram_bytes_per_chip: usize,
    pub activation_bytes_per_sequence: usize,
    pub sequences_resident_in_sram: usize,
    pub provenance: &'static str,
    pub no_speed_claim: bool,
}

How Qwen3.8 is served on a chip’s scale-up domain at one context length — every choice read off the chips’ memory hierarchy and topology and the model’s structure, none typed. provenance: modeled; no field is a time.

Fields

tensor_parallel: usize

The chips one tensor-parallel group spans (qwen38_tensor_parallel).

data_parallel: usize

The chips of the scale-up domain (default_tensor_parallel) the group does not span, as whole replicas.

ctx: usize

hbm_bytes_per_chip: usize

One chip’s HBM, its slice of the weights, and ONE sequence’s carried state on it at ctx: the KV cache (sharded over the KV heads, replicated beyond them), the Gated-DeltaNet memory and the conv rings.

weights_per_chip_bytes: usize

sequence_bytes_per_chip: usize

max_sequences_per_replica: usize

The most sequences a replica holds at once: their carried state beside the weights in every chip’s HBM.

max_sequences: usize

sram_bytes_per_chip: usize

One chip’s SRAM, one sequence’s per-step activation working set, and the most sequences whose working set — with their recurrent state — stays on-chip for a step.

activation_bytes_per_sequence: usize

sequences_resident_in_sram: usize

provenance: &'static str

no_speed_claim: bool

Trait Implementations

impl Clone for ServingPlan

fn clone(&self) -> ServingPlan

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ServingPlan

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for ServingPlan

fn eq(&self, other: &ServingPlan) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for ServingPlan

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Eq for ServingPlan

impl StructuralPartialEq for ServingPlan