Skip to main content

recognize_swiglu

Function recognize_swiglu 

Source
pub fn recognize_swiglu(
    stmts: &[Stmt<'_>],
    _interner: &Interner,
) -> Result<SwiGluSpec, ReductionReject>
Expand description

Recognize the SwiGLU idiom the sugar desugars to — Push (gate[iv] / (1 + E^(0 - gate[iv]))) * up[iv] in a counted loop. Backend-agnostic (CUDA + Tenstorrent).