Module cxx 

The libstdc++ behaviors llama.cpp’s vocabulary inherits, ported step for step.

sort is std::sort: llama.cpp orders its special-token cache by length alone, so tokens of equal length end in the order this introsort leaves them, and that order decides which of two overlapping special tokens of one length a text splits on first. unordered_map_order is the iteration order of a std::unordered_map<std::string, int> filled one key at a time: llama.cpp takes the first token its map yields that matches a list of texts as the end-of-turn and fill-in-the-middle tokens, so with two candidates (Qwen’s <|endoftext|> and <|im_end|>) the map’s order picks one. model_tokenizer_lock holds both equal to the library on this box.

Functions

hash_bytes

std::_Hash_bytes for 64-bit size_t (MurmurHash64A over unaligned little-endian words), the hash std::hash<std::string> runs with seed 0xc70f6907.

hash_string

std::hash<std::string>.

sort

std::sort(v.begin(), v.end(), less).

unordered_map_order

The order a std::unordered_map iterates keys inserted in the order of hashes (each key distinct): libstdc++’s _Hashtable with cached hash codes, a new node at the front of its bucket (or of the whole list when its bucket is empty), and the unique-key rehash that relinks nodes front-first. The result lists insertion indices.