Skip to main content

best_serving

Function best_serving 

Source
pub fn best_serving(
    h: &GgufHeader,
    card: &Card,
    s: &Serving,
) -> Result<ServingCost, ServingRefusal>
Expand description

The batch at which s emits the most tokens per second on card, and its pass.