Skip to main content

gemm_blk_expert_rows

Function gemm_blk_expert_rows 

Source
pub fn gemm_blk_expert_rows(choices: usize, experts: usize) -> usize
Expand description

The activation rows of a grouped GEMM block over an expert mixture’s choices: 128 when the average expert has more than 64 rows to multiply (choices / experts), 64 otherwise — a taller tile reads each expert’s decoded weights for more rows, a shorter one leaves fewer of its rows empty.