Skip to main content

rmsnorm

Function rmsnorm 

Source
pub fn rmsnorm(
    fp: FpUnit,
    x: VReg,
    w: VReg,
    out: VReg,
    t: VReg,
    n: usize,
    eps: f32,
) -> Vec<Op>
Expand description

RMSNorm of v[x] by the weights v[w] into v[out], n the vector’s length: rsqrt(Σx²·(1/n) + ε), then each x·r·w, 1/n the reciprocal at fp held as a constant. Scalars 0..2 are its scratch, v[t] too.