Skip to main content

gdn_gates

Function gdn_gates 

Source
pub fn gdn_gates(
    alpha: SReg,
    dt: SReg,
    a: SReg,
    beta: SReg,
    log_decay: SReg,
    decay: SReg,
    gate: SReg,
) -> Vec<Op>
Expand description

A Gated-DeltaNet head’s decay and gate from the scalars s[alpha], s[dt], s[a], s[beta]: the decay exponent softplus(α + dt)·a into s[log_decay], the decay exp of it into s[decay], the gate σ(β) into s[gate].