Skip to main content

compile_to_cuda_forward_holding

Function compile_to_cuda_forward_holding 

Source
pub fn compile_to_cuda_forward_holding(
    source: &str,
    weights: &HashMap<String, (String, usize)>,
    q8: &HashMap<String, Q8Ref>,
    moe: &HashMap<String, MoeWeights>,
    kind: Q8MatKind,
    held: &[&str],
) -> Result<String, String>
Expand description

The older forward of source at kind’s numerics with the optimizer’s steps held held back: each kernel it would fuse emitted apart, the program the fused one is held equal to.