Skip to main content

check_tp

Function check_tp 

Source
pub fn check_tp(spec: &ModelSpec, tp: usize) -> Result<(), NotYet>
Expand description

Can spec be split over tp chips? At any width of at least one chip (share), when every attention is unbiased and sinkless and every FFN is dense (its columns split; an expert mixture would split by experts).