Struct Streaming 

pub struct Streaming {

    pub set: ColumnSet,
    pub layout: ImageLayout,
    pub recurrence: bool,
    pub recurrence_width: usize,
    pub recurrence_lanes: usize,
    pub log_lines: usize,
    pub recurrence_flushes: usize,
    pub recurrence_credit: usize,
    pub vectors: bool,
    pub attention: bool,
    pub hard_float: Option<HardFloat>,
    pub float_rows: usize,
    pub vector_passes: usize,
    pub vector_depth: usize,
    pub contexts: usize,
    pub folds: usize,
    pub delta_norm: bool,
    pub conv_lanes: usize,
    pub conv_beats: usize,
}

The streaming unit of a hybrid engine (super::streaming::streaming_verilog), in the fast clock beside the engine’s: the column set it is built for and the image layout its matrices lie in on the channels, whose stripe is the row pairs a channel’s share of a stripe holds; and whether the recurrence unit and the vector unit’s side stand beside it.

Fields

set: ColumnSet

layout: ImageLayout

recurrence: bool

Whether a lazily flushed memory’s token-rows run on the fast side too (logos_gdn_side on the channels through logos_fast_port, GdnJob): each row’s log entry and count, fold, read-out and flush at the channels’ rate in the user clock, the core handing the row over and waiting for it. The unit walks a head’s memory a row of whole lines at a time, so it serves heads whose binary16 rows are whole lines, a width a multiple of 32.

recurrence_width: usize

The sixteens of elements a clock the recurrence unit’s read-out and flush take: 1 half a binary16 line (logos_gdn_read_p, logos_gdn_flush_p), 2 a whole line (logos_gdn_read_w, logos_gdn_flush_w, the flush holding a head’s rows and running its dot and update passes beside each other).

recurrence_lanes: usize

The head lanes the recurrence unit runs a row’s value heads on: 1 the side alone; more a coordinator and that many lanes (logos_gdn_lanes, super::gdn::lanes_verilog), lane k taking value heads k, k + lanes, … through its own fast port, the lanes’ ports and the shared one on the channels at once (logos_channel_mux), so with four lanes and a head a granule each lane reads and writes a channel of its own.

log_lines: usize

The lines each lane’s copy of the log holds (super::super::machine::Layout::log_lines of the layouts the build runs), a lanes build only: a row whose log would take more faults the recurrence unit, which runs none of it.

recurrence_flushes: usize

The flush units each head lane splits a flushing head’s rows over, a lanes build at a binary16 line a clock only: each unit holds HD / flushes of the rows through the same passes over the log, so a flushing head’s passes take 1 / flushes of the clocks and every row flushes as one unit flushes it. 1 a unit a lane.

recurrence_credit: usize

The lines the recurrence’s coordinator and each head lane ask ahead of those they have taken, a lanes build only (super::gdn::LANE_CREDIT unless given): behind a memory that answers latency clocks after an ask, a lane reads at most credit / latency lines a clock, so a credit past the channels’ latency reads a line a clock.

vectors: bool

Whether the vector jobs run on the fast side too (logos_vector_side, super::vector_side, on the recurrence unit’s port, VectorSideJob): each job’s loads, micro-program and stores in the user clock on the user clock’s vector unit, the core handing the job over and waiting for it.

attention: bool

Whether the attention jobs run on the fast side too (logos_attend_side, super::attend, on the recurrence unit’s port, AttendJob): each token-row’s query heads quantized and attended over the int8 cache in the user clock, the core handing each row over and waiting for it; the core keeps no attention unit or quantizer.

hard_float: Option<HardFloat>

The card’s floating-point DSP block the fast side’s f32 multiply and add are built on (HardFloat), or None for both in logic.

float_rows: usize

The token-rows the vector unit’s side holds the inputs of while a float matrix’s weights stream past (super::vector_side::FM_ROWS unless given): 16 lanes a row, the weights read once for each group.

vector_passes: usize

The passes of a job whose passes stand alone (super::super::machine::VectorJob::grouped) the vector unit’s side runs at once, each pass’s registers rows and scalars of its own (logos_simd_p’s PASSES), so a pass’s reductions and functions fill the pipeline its one chain leaves idle: 1 a pass at a time; at most the add’s latency and one, the slots a round of the unit’s reductions interleaves.

vector_depth: usize

The passes of the deepest vector job it runs whole the vector unit’s side’s registers hold (Streaming::with_vector_depth): its unit that many times as deep as that job’s vectors, so a job whose vectors are as deep as the side runs in groups as a shallower job’s do. 1 a register a vector deep.

contexts: usize

The sequencer contexts sharing the units: each a core of its own over its own token-rows, its rows table and its bank of the token-row store, the matvec unit serving one context while the row units (the recurrence, vector and attention sides) serve another, so one context’s weights stream while another’s rows run. 1 for one sequencer.

folds: usize

The fold lanes the streaming unit folds a job’s sums into its output slot over (super::super::machine::Fold, super::streaming::streaming_verilog_folding): a beat each 16 / folds clocks, read from the slot, folded and written back; 0 a unit that writes each sum as it is.

delta_norm: bool

Whether a recurrence job’s gated norm (super::super::machine::DeltaNorm) is made on its head lanes (super::gdn::delta_normed_lanes): each lane’s read-outs normed by the job’s weight and gated by the SiLU of the row’s gate as they are made, so a step whose gated norms are folded into its recurrence jobs (super::super::machine::fold_delta_norms) runs on it, a vector job fewer a Gated-DeltaNet block.

conv_lanes: usize

The lanes a convolution job’s rows run through on the vector unit’s side’s path of its own (super::vector_side::conv_path_verilog), 0 for none, and the most beats of its channels a tap of the kernel the path holds.

conv_beats: usize

Implementations

impl Streaming

pub fn new(set: ColumnSet, layout: ImageLayout) -> Streaming

The unit for set over layout, which lays its streamed matrices in stripes.

pub fn with_conv_path(self, lanes: usize, beats: usize) -> Streaming

The unit with each convolution job (the machine’s conv job, four taps, its ring rows whole lines) run on the vector unit’s side’s path of its own: the kernel read once a job, each row’s ring rows and x streamed through lanes conv lanes (1, 2, 4, 8 or 16 channels a clock), its output and ring row written as they come; beats the most beats of a convolution’s channels, the kernel’s depth a tap. A job of any other shape runs on the vector unit, which holds CONV_PATH_VREGS registers; one whose loads, stores or instructions name a register past them faults.

pub fn conv_beats<'a>(specs: impl IntoIterator<Item = &'a ModelSpec>) -> usize

The beats of Streaming::with_conv_path for checkpoints of the specs specs: the most channels of any of their Gated-DeltaNet layers’ convolutions (the keys twice and the values), sixteen a beat; 1 for none.

pub fn with_recurrence(self) -> Streaming

The unit with the recurrence unit beside it.

pub fn with_recurrence_width(self, width: usize) -> Streaming

The unit with its recurrence unit taking width sixteens of elements a clock, 1 or 2.

pub fn with_recurrence_lanes(self, lanes: usize, log_lines: usize) -> Streaming

The unit with its recurrence unit’s value heads on lanes head lanes, 1 to 4, each lane’s copy of the log log_lines lines.

pub fn with_recurrence_flushes(self, flushes: usize) -> Streaming

The unit with each head lane’s flushing head split over flushes flush units by rows, 1, 2, 4 or 8.

pub fn with_recurrence_credit(self, lines: usize) -> Streaming

The unit with its recurrence’s coordinator and head lanes asking up to lines lines ahead of those they have taken, from super::gdn::LANE_CREDIT to super::gdn::LANE_CREDIT_MAX.

pub fn with_vectors(self) -> Streaming

The unit with the vector unit’s side beside it, on the recurrence unit’s port onto the channels.

pub fn with_float_rows(self, rows: usize) -> Streaming

The unit with its vector unit’s side holding rows token-rows of a float matrix’s inputs at once.

pub fn with_vector_passes(self, passes: usize) -> Streaming

The unit with its vector unit’s side running a grouped job’s passes passes at a time, 1 to VECTOR_PASSES_MAX: a reduction’s passes in subgroups of the add’s latency and one, a slot of its round each.

pub fn with_vector_depth(self, depth: usize) -> Streaming

The unit with its vector unit’s side’s registers depth passes of the deepest vector job it runs whole deep, 1 to VECTOR_PASSES_MAX: a job whose vectors are that deep runs depth of its passes a group (up to the side’s passes a group), its chunks and its float matrices’ held inputs still a vector deep.

pub fn with_attention(self) -> Streaming

The unit with the attention side beside the vector unit’s side, on the recurrence unit’s port.

pub fn with_contexts(self, contexts: usize) -> Streaming

The unit with contexts sequencer contexts, 1 or 2, sharing its units: every unit must be on the fast side (the attention side beside the rest), since only the fast side’s units run beside one another, and a context’s program counter crosses to the shell in 16 bits.

pub fn with_hard_float(self, hard: HardFloat) -> Result<Streaming, HardError>

The unit with its fast side’s f32 adds, and its multiplies given a product block, on the floating-point DSP block hard: refused when a wrapper cannot give the soft unit’s latency on it, or the engine’s add, or the multiply of hard’s unit (HardFloat::product_target); a build of the card’s products has its product block.

pub fn with_folds(self, lanes: usize) -> Streaming

The unit folding a job’s sums into its output slot over lanes fold lanes, 1, 2, 4, 8 or 16: a step whose residual adds and gated products are folded into its matrix jobs (super::super::machine::fold) runs on it.

pub fn with_delta_norm(self) -> Streaming

The head lanes making each recurrence job’s gated norm as its read-outs are made (Streaming::delta_norm): a step whose gated norms are folded into its recurrence jobs runs on it.

pub fn signature(&self) -> String

The name the build and the programs written for it give the unit (LOGOS_D5005_STREAMING, engine.txt’s streaming): its column set, the row pairs a channel’s stripe holds, whether it folds, and whether its recurrence makes its gated norms.

pub fn pairs(&self) -> u64

The row pairs a channel’s share of a stripe holds: the column array’s cores a channel.

Trait Implementations

impl Clone for Streaming

fn clone(&self) -> Streaming

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Streaming

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Streaming

fn eq(&self, other: &Streaming) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Streaming