pub fn prefill(
spec: &ModelSpec,
dir: &str,
ctx: usize,
toks: &[usize],
pos0: usize,
) -> AuthoredExpand description
A PREFILL CHUNK: the tokens toks of ONE sequence at positions pos0, pos0 + 1, … (1-based) as one step, the
next token after the last of them out. The hidden states are stacked into a T × d matrix, so every projection,
norm, FFN and residual reads its weights once for the chunk; the chunk’s keys and values append to the cache as
one block from pos0, and the rows attend as one causal block. The state is the decode step’s own, so a decode
step continues where the chunk stops; the LM head reads the last row only.