Skip to main content

emit_qwen38_generate_batched

Function emit_qwen38_generate_batched 

Source
pub fn emit_qwen38_generate_batched(
    cfg: &Qwen38Cfg,
    wdir: &str,
    prompt: &[usize],
    n_gen: usize,
    batch: usize,
    hl: bool,
) -> String
Expand description

Emit the BATCHED decode program. batch sequences advance in lockstep, each with its own recurrence memory, conv ring and KV cache, sharing one pass over the weights.

The prompt is the same for every sequence on purpose: it makes the program self-checking. Batch-1 already has a known golden token stream, so every sequence in the batch must reproduce it — a batching bug that crosses sequences shows up immediately as one row diverging from the other B-1.