Skip to main content

attn_splits

logicaffeine_compile::codegen_cuda::lane::kernels

Function attn_splits 

Source 
pub fn attn_splits(blocks: usize, span: usize, sms: usize) -> usize
Expand description

The most chunks an attention cuts a row’s span into: enough blocks for two a streaming multiprocessor (sms) over the blocks the head blocks of one row give, and no more than give each chunk ATTN_MIN_SPLIT positions of span.