Skip to main content

axpy_staged

Function axpy_staged 

Source
pub fn axpy_staged(ty: GgmlType, numerics_mode: u32) -> bool
Expand description

Whether a per-head matmul whose rows are its inputs stages them (lk_head_axpy_staged, AXPY_W sub-blocks and AXPY_R row groups a block): under the quantized numerics in every block format but Q6_K and Q8_0 — 1.7–2.8 times the per-lane kernel on the 3060 there (19.9 µs to 9.7 for DeepSeek-V2-Lite’s Q4_K absorption); the per-lane kernel keeps the precise numerics’ f64 sums, Q6_K’s heavier decode and Q8_0’s one activation row. The choice reads no count of activation rows: a prompt’s chunk and a decode step sum alike, so a prompt continues alike in chunks or a token at a time.