pub fn compile_to_cuda_forward_q8_dp4a(
source: &str,
weights: &HashMap<String, (String, usize)>,
q8: &HashMap<String, Q8Ref>,
) -> Result<String, String>Expand description
compile_to_cuda_forward_q8 emitting the INT8-activation DP4A MMQ kernel for each Q8 matmul: the
activation is quantized to int8 per 32-block and __dp4a’d against the int8 weight (int32 accumulate) —
the same kernel shape llama.cpp’s fast path uses, and the shipped batch-1 default. Numerically the
int8-activation approximation of the dequant GEMV (small quantization error), not bit-identical; the
TwoPassInt8 mode recovers ~14-bit activation exactness at two __dp4a per block.