pub fn best_serving(
h: &GgufHeader,
card: &Card,
s: &Serving,
) -> Result<ServingCost, ServingRefusal>Expand description
The batch at which s emits the most tokens per second on card, and its pass.
pub fn best_serving(
h: &GgufHeader,
card: &Card,
s: &Serving,
) -> Result<ServingCost, ServingRefusal>The batch at which s emits the most tokens per second on card, and its pass.