Module safetensors 

Safetensors files as a checkpoint directory publishes them: one file, or shards named by model.safetensors.index.json. A file is an 8-byte little-endian header length, a JSON header naming every tensor’s dtype, shape and byte range within the data section, then the data. Tensors are read by position, never mapped or read whole, so a checkpoint of any size opens in the time it takes to read its headers.

Structs

Entry

One tensor’s header entry: its dtype, its shape (outermost first, as PyTorch states it) and its byte range within its file’s data section.

Shard

One safetensors file.

Enums

Dtype

An element type a safetensors header names.

Functions

shard_paths

The shard files a checkpoint directory holds: the files its index names, else its one model.safetensors, else every *.safetensors file in it. A file the index names that is not there is refused by name.