Skip to main content

attn_smem_bytes

Function attn_smem_bytes 

Source
pub fn attn_smem_bytes(g: usize, hd: usize, splits: usize, acc: usize) -> usize
Expand description

The dynamic shared memory an attention block takes: its g query heads of hd (f32), then — in accumulators of acc bytes — their scores over a tile and each head’s maximum, sum and fold factor over each of splits chunks.