Skip to main content

attn_blocks_a_multiprocessor

logicaffeine_compile::codegen_cuda::lane::kernels

Function attn_blocks_a_multiprocessor 

Source 
pub fn attn_blocks_a_multiprocessor(
    blocks: usize,
    sms: usize,
    acc: usize,
) -> usize
Expand description

The blocks of an attention (lk_attention‘s MB) a multiprocessor holds at once, by the registers it leaves them: three for a launch of more blocks than the chip holds at two (blocks over sms), which come in waves, so occupancy bounds them — on the 3060, 5.7% faster for Llama-3.2-1B’s batch of 64 and 3.4% for Gemma-3-1B’s; two for a launch under one wave, whose blocks’ latency bounds it, and for the f64 sums (acc 8 bytes), which need the registers.