Skip to main content

delta_chunk_scratch

Function delta_chunk_scratch 

Source
pub fn delta_chunk_scratch(chunks: usize, hv: usize, d: usize) -> [usize; 5]
Expand description

The floats of the chunked delta rule’s scratch over chunks chunks of hv value heads d wide: each chunk’s decayed keys and queries (a value of the width at a time), its T·diag(β) and masked scores, and its rows’ decays to its end with its own.