pub fn from_gated_delta(
spec: &GatedDeltaSpec,
names: &StageNames,
dense: &BTreeSet<String>,
target: &SpatialTarget,
) -> Result<Vec<SpatialProgram>, SpatialReject>Expand description
The Gated DeltaNet step over hv value heads reading hk key heads at head dim d:
u = S·k, S ← α·S + β·(v − α·u)·kᵀ, o = S·q / √d, RMSNorm’d, scaled by GN, gated by silu(z).
★ THREE SECTIONS, BECAUSE THE MEMORY IS READ ONCE AND WRITTEN ONCE. Section 1 reads S to form the
prediction u; section 2 writes the new S from the old one and u; section 3 reads the new S
to form the output. Each boundary is a barrier, so “read the old memory, then write, then read the
new one” is enforced rather than hoped for, and StaleStateRead catches any scheduler that tries
otherwise. Fusing them would make a wrong answer indistinguishable from a right one.
★ A BLOCK OF ROWS IS THE STEP, ROW AFTER ROW. The delta rule is a recurrence — row r reads the memory row r − 1
wrote — so a block of tokens is its rows’ three sections in turn against the one memory, each reading its row of the
operands in place and writing its row of the output. dense names the block’s tables the program lays out as
matrices; the output is written in its layout.