Struct Indexer 

pub struct Indexer {
    pub heads: usize,
    pub head_dim: usize,
    pub budget: usize,
    pub block: usize,
    pub layers: Vec<usize>,
}

Qwen’s sparse attention selection: heads index queries of head_dim over one index key a position, the keys of each complete block of block positions averaged, normed and rotated at the block’s first position, each block scored by Σ_h relu(q_h·k̄)/√head_dim; a query attends the best budget / block blocks and the positions after the last complete block, every visible position while the complete blocks are no more than that. The attention layers in layers select; any other attends every position.

Fields

heads: usize

head_dim: usize

budget: usize

block: usize

layers: Vec<usize>

Trait Implementations

impl Clone for Indexer

fn clone(&self) -> Indexer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Indexer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Indexer

fn eq(&self, other: &Indexer) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for Indexer

impl StructuralPartialEq for Indexer