Expand description
A checkpoint presented in the layout the spec’s arms read.
Qwen3-Next and Qwen3.5 compute one Gated DeltaNet, but a Qwen3-Next checkpoint (llama.cpp’s qwen3next, as its
converter writes it) pairs its value heads with its key heads grouped — value head k·r + i reads key head k, r
value heads a key head — and keeps each key head’s betas and alphas in one projection, ssm_ba. Qwen3.5’s tiles them
— value head t reads key head t mod k — and keeps ssm_beta and ssm_alpha apart. presented gives a
Qwen3-Next checkpoint Qwen3.5’s layout: every value-head axis reordered so tiled position t holds grouped head
(t mod k)·r + ⌊t/k⌋, the head its key head pairs it with either way, and ssm_ba split. The spec, the author, the
binder, the reference and every lowering then read one Gated DeltaNet; the spec refuses a checkpoint still holding
ssm_ba, so a grouped layout never reaches them unpresented.
A Phi-3 or GLM-4 checkpoint (llama.cpp’s phi3, glm4) keeps each block’s gate and up projections as one ffn_up
twice feed_forward_length wide, the gate’s rows first; presented gives it ffn_gate and ffn_up apart, as
every other dense SwiGLU stores them, and the spec refuses such a checkpoint read raw.
A GLM-4.5 checkpoint (llama.cpp’s glm4moe) stores each block’s FFN norm as post_attention_norm, the name other
families give the norm on the attention’s output; presented names it ffn_norm, and the spec refuses such a
checkpoint read raw.
Functions§
- presented
cpresented in the layout the spec’s arms read: a Qwen3-Next checkpoint in Qwen3.5’s, a Phi-3 or GLM-4 one with its gate and up projections apart, a GLM-4.5 one with its FFN norms namedffn_norm, every other as it is.