pub fn demonstrated_stage_latency(
step: &DecodeStep,
machine: &Machine,
published_tok_s: f64,
precision: Precision,
) -> Result<f64, DecodeRefusal>Expand description
The per-stage latency a published per-user rate implies: everything a token took beyond its memory floor, spread over its dependent stages. A rate this machine reached is a latency it has shown.