Module ngram_hash 

The row ids of Qwen3.8-Flash-Next’s hashed n-gram embedding (qwen4_exp‘s per-layer embedding), as transformers’ Qwen4ExpTextNGramEmbedding computes them.

For each position and each n-gram order n from 2 to ngram_size, the token and the n − 1 before it are each multiplied by the layer’s multiplier for their place and the products exclusive-ored; the result, reduced by each of the order’s heads_per_ngram heads’ primes and offset into that head’s block, names one table row per head. A token before the start of its segment (the end-of-sequence token ends a segment) reads as the end-of-sequence token. The multipliers are odd, drawn by splitmix64 from the layer’s seed, and small enough that every product of a token and a multiplier fits in a signed 64-bit integer; the heads’ primes are the consecutive primes after ngram_vocab_size_base, and the table is padded to a multiple of make_ngram_vocab_size_divisible_by rows.

Structs

NgramHash

One layer’s n-gram hash.