Struct MoeSpec 

pub struct MoeSpec {

    pub out: Symbol,
    pub x: Symbol,
    pub wr: Symbol,
    pub wg: Symbol,
    pub wu: Symbol,
    pub wd: Symbol,
    pub d: usize,
    pub dff: usize,
    pub ne: usize,
    pub k: usize,
    pub router_bias: Option<Symbol>,
    pub biases: [Option<Symbol>; 3],
    pub renormalize: bool,
    pub act: MoeAct,
    pub score: MoeScore,
    pub weigh_inputs: bool,
    pub selection_bias: Option<Symbol>,
    pub groups: Option<MoeGroups>,
    pub scale: Option<u64>,
}

A recognized Mixture-of-Experts FFN — Let out be moe of x with router wr gate wg up wu down wd choosing K of E. Lifted from the desugared block the SAME way recognize_matmul/recognize_rope lift theirs — purely structural, no markers, no naming conventions: the OPERAND tensors are the indexed reads NOT Let-bound inside the block (the temps are), and the dims fall straight out of the weight index arithmetic (wr[(t-1)·E+e] yields E; wg[((e-1)·dff+(o-1))·d+t] yields d and dff); out is the tensor the weighted accumulation writes.

Fields

out: Symbol

x: Symbol

wr: Symbol

wg: Symbol

wu: Symbol

wd: Symbol

d: usize

dff: usize

ne: usize

k: usize

router_bias: Option<Symbol>

The router’s bias, added to each logit before the softmax (router Wr plus Br).

biases: [Option<Symbol>; 3]

The experts’ gate, up and down biases (gate Wg plus Bg, …), each a row per expert.

renormalize: bool

The kept experts’ weights are renormalized to sum to one; false (without renormalizing) weighs each by its softmax over every expert.

act: MoeAct

The experts’ gated activation.

score: MoeScore

How the experts are scored: the softmax over every logit, or each logit’s own sigmoid.

weigh_inputs: bool

A kept expert’s weight scales the input it reads, and its output joins the sum unweighted (Llama 4).

selection_bias: Option<Symbol>

The selection bias (biased by B): each expert chosen by its score plus B[e], weighed by its score alone.

groups: Option<MoeGroups>

The group limit (within the top G of N groups).

scale: Option<u64>

The routed scale (scaled by S) as its bits: every kept weight times S, after the renormalization.

Implementations

impl MoeSpec

pub fn scale(&self) -> f64

The routed scale, one when the mixture has none.

pub fn weighs_outputs_by_softmax(&self) -> bool

Whether the router is a softmax over every expert weighting the kept experts’ outputs — chosen by their scores alone, among every expert, unscaled: the routing a lowering that knows no other computes.

pub fn sum_can_clamp(&self) -> bool

Whether the renormalizing sum’s clamp at 2⁻¹⁴ can bind: a softmax’s kept weights sum to at least K/E, so only a sigmoid’s — or a softmax over 16,384 experts or more — can fall below it.

Trait Implementations

impl Clone for MoeSpec

fn clone(&self) -> MoeSpec

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MoeSpec

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for MoeSpec

fn eq(&self, other: &MoeSpec) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for MoeSpec

impl StructuralPartialEq for MoeSpec