pub fn lower_batched_tp_decode_streamed(
spec: &ModelSpec,
storage: Storage<'_>,
ctx: usize,
batch: usize,
tp: usize,
target: &SpatialTarget,
fill: impl Fn(&str) -> String,
) -> Result<StreamedModel, StreamedReject>Expand description
Lower spec as an SN50 BATCHED, TENSOR-PARALLEL decode deployment — batch sequences per step over tp chips,
the serving shape: lower_batched_decode_streamed’s stacked programs with every layer kind
lower_tp_decode_streamed’s program over the chips.