pub fn gemv_mma_shape(mt: usize) -> (usize, usize, usize)Expand description
lk_gemv_mma’s shape over mt m-tiles, (nt, mw, nw): four n-tiles of 8 outputs a block over one m-tile, a warp
each; eight over two, a warp each n-tile over both; sixteen over four, a warp each two n-tiles over two m-tiles — so
a block stages its activation rows once for 32·mt outputs, and over four m-tiles each fragment a warp loads feeds
two products. On the 3060 at 64 rows the two-by-two warps beat a warp each n-tile over all four m-tiles on
Llama-3.2-1B’s down projection (8192 by 2048) 117 µs to 102, its output projection 33.8 to 30.9, its query, key and
value 35.6 to 32.9 and its gate and up 171.6 to 168.8; at 16 and 32 rows two n-tiles a warp gain nothing.