Function emit_qwen38_generate 

pub fn emit_qwen38_generate(
    cfg: &Qwen38Cfg,
    wdir: &str,
    prompt: &[usize],
    n_gen: usize,
    precise: bool,
    deq: u8,
) -> String

Emit a self-contained CUDA program that generates from the real Qwen3.8-27B weights and reports throughput.

Where the speed comes from

A dense 27B at Q8_0 reads ~27 GB of weights per token, so batch-1 decode is bandwidth-bound and the GEMV is ~90% of the work. Three things follow, and the emitter is built around them:

1. One GEMV per activation, not four. extract_qwen38 concatenates every projection that shares an activation into a single matrix (qkvz, qgkv, gu), so a GDN layer’s q/k/v/z is ONE launch over 16384 output rows with no branching, and the FFN’s gate+up is one launch over 34816.
2. One CUDA graph for the whole token. ~800 kernels per token at a few µs of launch overhead each is milliseconds against a ~24 ms budget. The decode step is captured once and replayed, with the sampled token and the position advanced ON DEVICE (gk_argmax_adv) so no replay needs a host round trip.
3. Fusion past what a per-operator graph can express. The conv carries its silu, both GDN gates come from one launch, the recurrence touches its memory once (see gdn.rs), and the residual add, the next RMSNorm and the activation re-quantization are a single kernel.

precise swaps the split-K flash attention for the byte-exact multi-pass kernel — slower, but it does not reorder float additions, so it is the lane to check a token stream against llama.cpp with. deq selects how much of the decode keeps its activations in f32 instead of quantizing them to int8: 0 = none (dp4a everywhere), 1 = the LM head only, 2 = every Q8_0 GEMV.

At batch 1 this is nearly free. A GEMV reads a whole weight matrix and ONE activation vector, so the activation contributes no measurable traffic and quantizing it saves no bandwidth — it only lets the inner loop use dp4a. The dequantizing form costs ~3 float ops per weight instead of a quarter of a dp4a, which at 1400 GB/s works out to about 4 TFLOP/s against the A100’s 19.5: still memory bound. What it buys is the removal of the activation quantization error entirely.