Struct Layout 

pub struct Layout {
    pub numerics: EngineNumerics,
    pub tensors: Vec<Placed>,
    pub rotary: Vec<(Rope, u64)>,
    pub ctx: usize,
    pub regions: u64,
    pub region: u64,
    pub sequences: usize,
    pub blocks: Vec<BlockState>,
    pub bytes: u64,
}

The card’s memory as a machine lays it out.

Fields

numerics: EngineNumerics

tensors: Vec<Placed>

Every tensor the program reads, in Program::tensors order.

rotary: Vec<(Rope, u64)>

Every rotary table the program turns heads by, in Program::ropes order, each at its address: per position, dims/2 cosines then dims/2 sines, f32.

ctx: usize

Positions every sequence may reach.

regions: u64

Where the sequences’ regions start, each region bytes, sequences of them.

region: u64

sequences: usize

blocks: Vec<BlockState>

Each layer’s place in a region.

bytes: u64

Implementations

impl Layout

pub fn of( spec: &ModelSpec, numerics: EngineNumerics, program: &Program, source: &dyn WeightSource, sequences: usize, ctx: usize, ) -> Result<Layout, EngineError>

The memory of sequences sequences of spec at numerics, each reaching ctx positions, over the tensors program reads from source.

pub fn region_of(&self, s: usize) -> u64

Where sequence s’s region starts.

pub fn log_lines(&self) -> usize

The most lines a lazily flushed memory’s log holds past its count’s line, every entry it holds at its longest: none for a layout of no such memory.

Trait Implementations

impl Clone for Layout

fn clone(&self) -> Layout

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Layout

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Layout

fn eq(&self, other: &Layout) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Layout