Skip to main content

attn_flash_queries

logicaffeine_compile::codegen_cuda::lane::kernels

Function attn_flash_queries 

Source 
pub fn attn_flash_queries(hd: usize, vd: usize) -> usize
Expand description

The query vectors a block of the flash attention (lk_attention_flash) takes: 64 for heads and values to 128 wide, 32 past them, so a thread’s outputs stay 64 registers and the block’s shared memory within a block’s.