Skip to main content

tp_decode_step

Function tp_decode_step 

Source
pub fn tp_decode_step(
    spec: &ModelSpec,
    dir: &str,
    ctx: usize,
    tok: usize,
    pos: usize,
    tp: usize,
) -> Result<TpAuthored, NotYet>
Expand description

A whole decode step over tp chips: the embedding lookup and the head replicated, every layer split.