Skip to main content

attn_run_width

logicaffeine_compile::codegen_cuda::lane::kernels

Function attn_run_width 

Source 
pub fn attn_run_width(hd: usize) -> usize
Expand description

The lanes a key’s dot takes in an attention of heads hd wide (lk_attn_run_width, lk_attention’s RW): a half warp for a head of 9 to 16 fours, the whole warp otherwise.