pub fn emit_moe_forward(x: &[f64], w: &MoeWeights) -> StringExpand description
Emit a self-contained CUDA program for ONE Mixture-of-Experts FFN block over real int8 weights: router GEMV
→ top-K route (softmax, keep K, renormalize) → each expert’s SwiGLU with an EXPERT-INDEXED weight base →
weighted combine. The weights stream from files (int8 quants + f32 block scales), never a 4×-larger f32
buffer, so the 30B block (~30 GB int8) fits a 48 GB card. Dequant-in-kernel GEMV (f32 activation × int8
weight, f64 accumulate) matches the LOGOS moe op on the VM exactly; the whole block is captured as one
CUDA graph. x is the block input (d floats, inlined). The experts are UNROLLED in the host launch list
(not the kernels), so E scales without changing a kernel — the same emitter serves 8B-scale and 30B-scale.