Module gdn 

Gated DeltaNet lowered to CUDA — the mechanism behind 48 of Qwen3.8-27B’s 64 layers.

Each value head carries a fixed D×D associative memory S and rewrites it per token:

  u = S·k                      what the memory predicts for this key
  S = α·S + β·(v − α·u)·kᵀ     decay, then correct by the prediction error
  o = S·q                      read the UPDATED memory
  o = rmsnorm(o ⊙ silu(z))·GN  gate FIRST, then normalize

The fusion

Written literally that is three passes over S — one for S·k, one for the rank-1 update, one for S·q. A dataflow graph of separate operators (ggml’s shape) cannot do better, because each operator’s output is a materialized tensor. But the update is affine in S, so the second matvec expands:

  o[a] = Σ_b S_new[a][b]·q[b]
       = Σ_b (α·S_old[a][b] + coef[a]·k[b])·q[b]
       = α·(S_old·q)[a]  +  coef[a]·(k·q)

(k·q) is ONE SCALAR per head. So a single pass that loads S_old[a][b] into a register accumulates S·k, S·q and k·q together, and the register is still live for the write-back — one read and one write of the memory per token, with the post-update read costing nothing extra. That is the whole reason this op is a first-class LOGOS block rather than a composition of smaller ones.

Constants

GDN_KERNELS

One block per (value head, row) computes that row’s u, w and the shared k·q in a single pass over the memory, then rewrites the row from the register it already holds. o lands in scratch; the gated RMSNorm needs the whole head, so it is a second (tiny) kernel.

QWEN38_KERNELS

The rest of the Qwen3.8-specific device code: the pieces of a GDN layer that surround the recurrence, plus the two things gated attention needs that a plain transformer does not (a partial rotary and an output gate). Each is deliberately fused past what a per-operator graph could do — the conv carries silu, the gate kernel computes both gates from one launch, and the norm+rope stay one pass over the head.

Functions

emit_gated_delta_forward

Emit a self-contained CUDA program for ONE Gated DeltaNet step with every input inlined, so the fused kernels above can be checked against the LOGOS gated delta op on the VM at small shapes on any GPU — the same shape of proof emit_moe_forward gives the MoE block. Prints O then the mutated memory S, because a desugar that ignored the memory would otherwise pass.