Skip to main content

ATTN_I8_QUERIES

logicaffeine_compile::codegen_cuda::lane::kernels

Constant ATTN_I8_QUERIES 

Source 
pub const ATTN_I8_QUERIES: usize = 64;
Expand description

The query vectors a block of the two-word attention (lk_attention_i8) takes, a warp 16 of them.