Struct Gguf 

pub struct Gguf {
    pub version: u32,
    pub meta: BTreeMap<String, MetaValue>,
    pub order: Vec<String>,
    pub arch: Arch,
    /* private fields */
}

A parsed GGUF file: the metadata, the tensor directory, and where the data section starts.

Fields

version: u32

meta: BTreeMap<String, MetaValue>

order: Vec<String>

The metadata keys in the order the file lists them.

arch: Arch

Implementations

impl Gguf

pub fn open(path: &Path) -> Result<Gguf>

Open and parse a GGUF file’s header, metadata and tensor directory. The weights stay on disk until read.

pub fn data_start(&self) -> u64

Where the data section starts: a tensor’s bytes sit at data_start() + offset.

pub fn header_digest(&mut self) -> Result<u64>

The FNV-1a 64 digest of everything before the data section — the header, the metadata and the tensor directory: a program compiled against this checkpoint checks it reads the same file.

pub fn meta(&self, key: &str) -> Option<&MetaValue>

A metadata value by its full key.

pub fn arch_meta(&self, suffix: &str) -> Option<&MetaValue>

A metadata value under the architecture prefix ({arch}.{suffix}).

pub fn decode_read_bytes(&self) -> Result<u64>

The bytes of weights one decode step reads for a token. That is every tensor of the model’s blocks and around them, with three exceptions:

• the token embedding, where an output matrix of its own stands beside it (the step reads one of its rows);
• each expert matrix, counted in the share of its experts a token is routed to (expert_used_count of expert_count);
• the draft blocks appended for multi-token prediction, which count nothing. A tensor in a format this crate does not read is an error.

pub fn tensors(&self) -> &[TensorInfo]

The tensor directory, in file order.

pub fn info(&self, name: &str) -> Option<&TensorInfo>

One tensor’s directory entry.

pub fn tensor_type(&self, name: &str) -> Result<GgmlType>

The format of a tensor, or why it is not one this crate reads.

pub fn row_bytes( &mut self, name: &str, first: usize, count: usize, ) -> Result<Vec<u8>>

The raw bytes of rows [first, first + count) of a tensor, a row being its dims[0] fastest elements.

pub fn dequant_rows( &mut self, name: &str, first: usize, count: usize, ) -> Result<Vec<f32>>

Rows [first, first + count) of a tensor dequantized to f32, whatever its format.

pub fn dequant_tensor(&mut self, name: &str) -> Result<Vec<f32>>

A whole tensor dequantized to f32, whatever its format.

pub fn dims2(&self, name: &str) -> Option<(usize, usize)>

(in = ne0, out = ne1) for a 2-D weight tensor (y = W·x uses out×in).

pub fn dims(&self, name: &str) -> Option<&[u64]>

The [ne0, ne1, …] dims of a tensor (ggml order: ne0 is the fastest / in-features).

pub fn tensor_names(&self) -> Vec<(String, Vec<u64>, u32)>

Every tensor’s (name, dims, ggml_type) — for inspecting an unfamiliar checkpoint’s layout.

pub fn has(&self, name: &str) -> bool

true if the named tensor exists.

pub fn f32_tensor(&mut self, name: &str) -> Result<Vec<f32>>

Read an F32 tensor’s data in full (the RMSNorm weights).

pub fn f16_tensor(&mut self, name: &str) -> Result<Vec<f32>>

Read an F16 tensor, widened to f32. The Qwen3.8 vision tower (mmproj-*.gguf) stores every weight as F16.

pub fn float_tensor(&mut self, name: &str) -> Result<Vec<f32>>

Read an F32 or F16 tensor as f32, whichever the checkpoint stored.

pub fn q8_0_matrix(&mut self, name: &str) -> Result<(Vec<i8>, Vec<f32>)>

Read a whole Q8_0 tensor as (quants i8 [out*in] row-major, scales f32 [out*in/32]). dequant W[o][t] = quants[o*in+t] * scales[o*nb + t/32]. Works for 2-D [in, out] AND 3-D expert stacks [in, out_per_expert, n_expert] — for the latter out folds to out_per_expert · n_expert, so ffn_gate_exps/ffn_up_exps come out as the stacked [E·dff, d] (and ffn_down_exps as [E·d, dff]) layout emit_moe_forward consumes, expert e’s rows at e·dff.

pub fn q8_0_dequant_row(&mut self, name: &str, row: usize) -> Result<Vec<f32>>

Dequantize a single Q8_0 row to f32 (a token embedding row).

pub fn q8_0_tile_blocks( &mut self, name: &str, rows: usize, cols: usize, ) -> Result<(Vec<i32>, Vec<f32>)>

Read a rows × cols Q8_0 sub-tile as (quants, scales): the int8 values (rows*cols, row-major) AND the per-32-block f32 scales (rows * cols/32). The full Q8_0 weight (dequant = quant * scale), the form a real block-scaled GEMM consumes.

pub fn q8_0_tile( &mut self, name: &str, rows: usize, cols: usize, ) -> Result<Vec<i32>>

The raw int8 weights of the first rows × cols sub-tile of a Q8_0 2-D tensor, row-major over the tensor’s ggml layout. cols must be ≤ ne0 and a multiple of 32, rows ≤ ne1.

Trait Implementations

impl SpecSource for Gguf

fn arch(&self) -> &str

general.architecture.

fn arch_meta(&self, suffix: &str) -> Option<&MetaValue>

The architecture key {arch}.{suffix}.

fn dims(&self, name: &str) -> Option<&[u64]>

A tensor’s dims, dims[0] the fastest-varying.

fn tensor_names(&self) -> Vec<&str>

Every tensor’s name, in the directory’s order.

fn has(&self, name: &str) -> bool

impl WeightSource for Gguf

fn describe(&self, name: &str) -> Result<(GgmlType, Vec<u64>)>

A tensor’s format and dims.

fn row_bytes( &mut self, name: &str, first: usize, count: usize, ) -> Result<Vec<u8>>

The bytes of rows [first, first + count), a row being the tensor’s dims[0] elements.

fn has(&self, name: &str) -> bool