Skip to main content

qwen38_serving_plan

Function qwen38_serving_plan 

Source
pub fn qwen38_serving_plan(
    cfg: &Qwen38Cfg,
    facts: &RduChipFacts,
    ctx: usize,
) -> Result<ServingPlan, String>
Expand description

Qwen3.8’s serving plan on facts’ chips at context ctx: the tensor-parallel width the model’s heads admit within the scale-up domain, the rest of the domain as data-parallel replicas, and how many sequences a replica holds — HBM-bound by each sequence’s carried state beside the chip’s weight slice, SRAM-resident as far as each sequence’s per-step working set allows. The per-chip bytes are the fit report’s own (qwen38_fit).