Module quant 

The GGUF weight formats, and their dequantization to f32.

Each format is a run of fixed-size blocks; a block holds a fixed count of weights and the scales that turn its packed integers back into numbers. The dequantizers here follow ggml’s own reference definitions (ggml/src/ggml-quants.c, dequantize_row_*) operation for operation, in the order those definitions write them, so their output is the reference’s to the bit (model_quant_lock compiles the reference and compares).

Structs

ScaleGrid

The scale of a scaled-float tensor (an FP8 checkpoint’s weight_scale_inv) broadcast over the weight’s elements, as the Hugging Face converter’s dequant_simple broadcasts it before weight.float() * scale: with a block size, each of the scale’s trailing axes repeated a block’s length and cropped to the weight’s; then axes of length 1 appended until the scale has the weight’s rank, every axis either 1 (broadcast) or the weight’s length.

Enums

GgmlType

A ggml tensor type this crate reads, by its GGUF type id.

Constants

KVALUES_IQ4NL

The non-linear levels an IQ4 nibble indexes (ggml’s kvalues_iq4nl).

Functions

bf16_to_f32

bfloat16 → binary32: the high half of the word.

e4m3fn_to_f32

An e4m3fn float’s value as torch widens it: bias 7, no infinities, S.1111.111 a NaN keeping its sign and mantissa bits.

e5m2_to_f32

An e5m2 float’s value: the high byte of a binary16.

f16_to_f32

IEEE-754 binary16 → binary32, exactly (subnormals included).

f32_to_bf16

x rounded to bfloat16, to nearest with ties to even; a NaN stays a quiet NaN.

f32_to_f16

x rounded to binary16, to nearest with ties to even: subnormals kept, past the largest finite value infinite, a NaN quiet.