Skip to main content

emit_scaled_int8_gemm

logicaffeine_compile::codegen_cuda::matrix

Function emit_scaled_int8_gemm 

Source 
pub fn emit_scaled_int8_gemm(
    m: usize,
    k: usize,
    n: usize,
    wq: &[i32],
    ws: &[f32],
    aq: &[i32],
    as_: &[f32],
) -> Result<CudaGemmArtifact, CudaLowerError>
Expand description

Emit a block-scaled Q8_0 GEMM CUDA program: DP4A int8 dot per 32-block, scaled by the baked f32 block scales, accumulated in f32. Prints O(m×n) row-major as %.9g (one per line). k must be a multiple of 32 (whole Q8_0 blocks). Refuses (typed) if the RDU-shaped target lacks int8 weight-stationary GEMM.