Module column 

The streaming engine’s matrix columns as Verilog: a matrix streamed block-major within groups of row pairs, each (row, token-row) still the row’s blocks added in order, so every result is crate::spatial::decode_engine::numerics::dot’s.

A column takes its channel’s weight stream a group of rows at a time and every row pair’s block b before any pair’s block b + 1. A block’s activations then serve every pair of the group in turn, and each (pair, token-row) sum comes round once a group’s pairs and token-rows have passed, which is what lets the block step be pipelined without reordering a row’s sum.

What every core of a column array would otherwise compute alike is computed once beside them. A beat (a pair’s two blocks) is decoded once, by logos_column_decode_<set>, into its blocks’ codes, their units’ multipliers and offset terms as bytes, their scales, and their units’ sums of codes; a token-row’s activation block is written to block RAM as each activation word’s excess codes and their units’ sums (logos_column_act_word), so a read serves every core.

logos_column_core_<set> is a column’s datapath: it rings decoded beats — the one its issues run at the ring’s head and those queued behind it, each put in as its feed decodes it — and takes the next by moving the head, so a step moves a pointer and never a beat. It runs each issue of the beat at the head against a token-row as passes, a clock each: one per 4-bit digit of the format’s codes and, with two activation words, per word, each pass the beat’s codes and the token-row’s word through one digit and one word of systolic processing elements (logos_matvec_pass_p, PASS_LATENCY clocks), its partials summed in their places as they leave them, the pass’s flags and the fields its lanes take delayed beside them. The issue’s last pass hands the sums to two pipelined lanes (logos_lane_value_p), and the value is added into its sum through the pipelined logos_fadd_p; a sum’s last block leaves the add as the sum’s emission while its place goes back to +0 for the next group. An issue whose sum is still in the add waits at its last pass.

logos_column_<set> is one core with its own sequencer over groups of row pairs: it takes the next beat the clock it issues the last token-row of the one it holds, so a block step of fewer passes than the add has stages takes FADD_LATENCY + 1 clocks and a larger one a clock a pass (column_clocks).

logos_column_array_<set> is the streaming engine’s matvec: CH · N cores, N on each memory channel, one row pair each of their channel’s tile of a stripe (super::super::image::ImageLayout::stripe), every core running the same pass of the same token-row of the same block step each clock against one activation read. A step issues when every core whose channel’s tile holds its pair has its beat; each channel’s logos_column_feed_<set> puts its cores’ decoded beats into their rings as it deals them, and the cores take the next step’s the clock the last token-row issues (column_array_clocks).

A set with octets (ColumnSet::octets) takes a Q8_0 matrix’s blocks eight at a time: a beat holds a row’s eight blocks (logos_dec_q8_0x8, 256 elements in 272 bytes), the processing elements run every unit of them in one issue, and the array keeps each issue’s sums in a slot of one of two buffers instead of handing them to the lanes. A full buffer’s blocks then leave for the lanes a block a clock — every token-row’s block 0, then every token-row’s block 1, and so on — each as the lone Q8_0 block an issue carries otherwise (its two units the lane’s first two, its scale and its activation block’s scale), and each the clock no add still holds its sum, so a sum’s blocks are added in order. While one buffer’s blocks leave, the next step’s issues fill the other: a Q8_0 block costs the lanes a clock instead of an issue of two passes.

logos_matvec_pass_x2 (matvec_pass_x2_verilog) is two cores’ passes on one set of processing elements in a clock of twice the cores’ (clk2, its rising edges on clk’s and between them): the cores of a channel run in lockstep against one activation word, so the elements take core a’s codes with the word at clk2’s edge between clk’s and core b’s at clk’s next, and each core’s partials leave as logos_matvec_pass_p gives them.

Structs

ActFields

The bit offsets of an activation word’s fields.

ColumnSet

A set of formats one column is built for, under one name, and the activation words its jobs run at: built for one word, every module of the set has no two and its lanes are one word wide. With octets, a Q8_0 matrix’s blocks come eight to one of the column’s (its octets, ColumnSet::block_len): an issue’s units hold a row’s eight blocks, and the lane takes them one a clock, each still a block of its own. With a slot past 1 every block comes in the image padded with zeros to a multiple of slot bytes (ImageLayout::slot), so the feed gathers its beats slot bytes at a time. With lanes past 1 an octet’s blocks leave for lanes lanes a row a clock, a token-row’s next lanes blocks at once: the row’s full lane and lanes − 1 lanes of one scale (logos_lane_slim_p), their values added into the row’s sum in block order through a chain of lanes adds.

DecodedFields

The bit offsets of a decoded block’s fields.

Constants

COLUMN_VALUE_LATENCY

Clocks from a block’s issue to its value at the column’s add: the activations’ read (the codes’ register beside it), the processing elements, the lanes’ input registers, and the lane.

PASS_LATENCY

Clocks from a pass’s operands to its partial sums in logos_matvec_pass_p: the systolic elements’.

Functions

column_array_clocks

The clocks from the edge that takes start to the edge that raises the array’s last sums, for streams that offer every core its beat whenever it is ready: a stripe’s block step is one pair a core against every token-row, so the array runs as one column of one pair a group over the stripes.

column_array_octet_clocks

column_array_clocks for an octet job (ColumnSet::octets): octets of a row’s eight blocks each a stripe’s block step. A step’s issues run a pass a clock once the step before has issued its last, each into its buffer’s slot, the first waiting for its buffer to be free; a buffer’s blocks are asked for from four clocks after its last issue, block-major, a clock each and no sooner than FADD_LATENCY + 1 clocks after its sum’s block before, and the buffer is free the clock after its last; the last block’s sum leaves as an issue’s would.

column_array_octet_lane_clocks

column_array_octet_clocks for a set of lanes octet lanes a row (ColumnSet::with_octet_lanes): a buffer’s blocks asked for lanes at once — a token-row’s next lanes — a clock each group, no sooner than lanes adds and a clock after its sum’s group before, the last group’s sum leaving through the chain’s lanes adds.

column_clocks

The clocks from the edge that takes start to the edge that raises the column’s last sum, for a stream that offers a beat whenever the column is ready and issues of passes passes: a clock to take the first beat; a block step (every pair’s beat of one block against every token-row) of pairs · tokens issues, a pass a clock, its first issue’s last pass at least FADD_LATENCY + 1 clocks after the step before’s, waiting for the sum that one left in the add; then the last issue’s value through the lane and the add.

column_passes

The passes an issue of a block of ty takes: one per 4-bit digit of its codes, for each activation word.

column_verilog

The columns for every format, each its own set for both word counts, with the block steps, decoders, lanes, f32 units and block RAM they are built from.

columns_verilog

The columns of each set, with the modules they are built from: logos_column_decode_<name>, logos_column_core_<name>, logos_column_<name>, logos_column_array_<name> and logos_column_feed_<name>. A set’s core holds a block of any of its formats, its widths the set’s widest; with more than one format each module takes the job’s format as fmt, its place in the set, a block of fewer elements than the widest running with the units past it zero.

matvec_pass_x2_verilog

logos_matvec_pass_x2: two cores’ passes, each logos_matvec_pass_p‘s at its latency (PASS_LATENCY), on one set of processing elements in a clock of twice the cores’.

matvec_pass_x2t_verilog

logos_matvec_pass_x2t: a core’s two token-rows’ passes, each logos_matvec_pass_p’s at its latency (PASS_LATENCY), on one set of processing elements in a clock of twice the core’s.