Skip to main content

attn_i8_scratch

logicaffeine_compile::codegen_cuda::lane::kernels

Function attn_i8_scratch 

Source 
pub fn attn_i8_scratch(
    len: usize,
    groups: usize,
    hd: usize,
    vd: usize,
) -> [usize; 3]
Expand description

The two-word attention’s scratch for a cache of len positions over groups key/value groups: the keys’ high and low words, the values’, and each row’s two scales — in bytes.