pub fn emit_qwen38_mtp_probe(
cfg: &Qwen38Cfg,
wdir: &str,
prompt: &[usize],
golden: &[usize],
) -> StringExpand description
Emit the MTP ACCEPT-RATE PROBE.
Multi-token prediction only pays if the draft head is usually right: the speedup is
(1+alpha) * full / (full + draft), and with the draft costing 7% of a forward that is 1.24x at
alpha=0.33 and 1.59x at alpha=0.70. The expensive half of MTP is the two-token VERIFY step — GEMVs batched
over two positions while the recurrence still advances sequentially — so it is worth measuring alpha
BEFORE building it, not after.
This probe does exactly that and nothing more. It walks the known golden continuation, and at each step runs the draft head on (last hidden state, token just produced) and asks whether it predicted the NEXT golden token. No verification path, no speculation machinery — just the accept rate, which is the number that decides whether the rest is worth writing.
The draft must not disturb the decode it speculates on: it has its own KV cache and uses an argmax that does not advance the position.