pub fn lower_batched_decode_streamed(
spec: &ModelSpec,
storage: Storage<'_>,
ctx: usize,
batch: usize,
target: &SpatialTarget,
fill: impl Fn(&str) -> String,
) -> Result<StreamedModel, StreamedReject>Expand description
Lower spec as an SN50 BATCHED decode deployment: batch sequences per step, each with its own token, position
and carried state, every weight read once for all of them (the model’s Cut::Batch). The same three kinds of
program as lower_decode_streamed — the embedding lookups, one program per layer kind, the head — over
the stacked batch × d hidden states; the token ids toks and each sequence’s pos_s{b} are runtime inputs.