Skip to main content

GEMM_PARTS

Constant GEMM_PARTS 

Source
pub const GEMM_PARTS: usize = 6;
Expand description

The matrices a launch in parts (lk_parts: lk_gemv_parts, lk_gemv_mma, lk_gemm_blk‘s G 4) multiplies one activation by at most — the kernels’ LK_PARTS: six, so a recurrent layer’s query, key, value, output gate and two gates project its activation in one launch, the gates no launch of their own (at Qwen3.5-0.8B’s batch of 64 a single block on the 3060, 16.8 µs a layer).