Skip to main content

attn_values_per_thread

Function attn_values_per_thread 

Source
pub fn attn_values_per_thread(vd: usize) -> usize
Expand description

The pairs of each head’s values a thread of an attention block (256 threads) accumulates: lk_attention weighs at most g times as many of its block’s g heads’ value slots a thread.