Skip to main content

decode_batch_head

Function decode_batch_head 

Source
pub fn decode_batch_head(spec: &ModelSpec, dir: &str, bsz: usize) -> Authored
Expand description

A batched decode step’s head alone: the last layer’s stacked output in, every sequence’s next token out.