pub fn decode_step(
spec: &ModelSpec,
dir: &str,
ctx: usize,
tok: usize,
pos: usize,
) -> AuthoredExpand description
ONE DECODE STEP: token tok at position pos (1-based) in, the next token out. Every attention layer appends
this token’s key and value to its KV cache (ctx positions, the step’s carried state alongside any recurrent
memories and conv rings) and attends over the first pos rows; the step ends with the arg-max of the logits.