Skip to main content

compile_to_cuda_forward_q8_moe

Function compile_to_cuda_forward_q8_moe 

Source
pub fn compile_to_cuda_forward_q8_moe(
    source: &str,
    weights: &HashMap<String, (String, usize)>,
    q8: &HashMap<String, Q8Ref>,
    moe: &HashMap<String, MoeWeights>,
) -> Result<String, String>
Expand description

compile_to_cuda_forward_q8 with a MoE weight manifest: any moe op in the source lowers to grouped-expert CUDA (router GEMV → top-K route → per-expert SwiGLU → weighted combine), its router/gate/up/ down weights loaded int8-resident from the files named in moe (keyed by the block’s output variable). This is the Qwen3-30B-A3B FFN — the SAME emitter as the dense 8B forward, plus the MoE stage.