Module streaming 

The streaming engine’s units composed from the columns, the feeds and the activation quantizer.

logos_streaming_matvec_<set> runs a machine matvec job (crate::spatial::decode_engine::machine::Job::Matvec) from the token-row store to the token-row store, its matrix laid in stripes and streamed from the four memory channels’ image streams. It quantizes each token-row’s input a weight block at a time, the store’s beats through logos_act_stream into the column array’s activations, then streams the stripes through the channels’ feeds into the array, and writes each clock’s sums — one stripe’s rows for one token-row — into the output slot’s row as they leave, so the output row is numerics::dot of each matrix row with the quantized input, the machine’s own values. A clock’s sums are queued in the stripe’s row order and written a store beat a clock: the 8N rows of a stripe of N pairs a channel span one beat or several, and a stripe of an odd N starts half a beat in when it is odd.

A job’s matrix comes off each channel as a TileRun: the channel’s tiles of the tensor, every one but the last the same lines and row bytes, a fixed number of lines and rows apart. logos_tile_run walks one into the channel’s logos_image_stream, which asks the channel for each tile’s lines and sends its rows on as beats.

Structs

StreamEntry

What a hybrid engine’s streaming unit takes of one streamed tensor beside its job, as the engine reads it from three lines of the card’s memory at the tensor’s address: line 0 the tensor’s format’s place in the unit’s set, its blocks a row, its rows, its stripes, each channel’s turn (two bits a channel from the lowest), its tag in the image and whether a job of it is greedy, a 32-bit word each from the lowest; lines 1 and 2 the four channels’ tile runs, channel c‘s from bit 256 c of the two lines’ 1024, eight words: its count, line, lines, bytes, last tile’s lines and bytes, first row and row step. A greedy job writes each token-row’s arg-max of the matrix’s rows and that row’s value, a beat a token-row, in place of the rows: a head’s. The greedy flag is bit 0 of line 0’s seventh word, and from bit 1 its two bits are how the job of the tensor folds each sum into its output slot (Fold::code): a step uses each matrix in one job.

TileRun

One channel’s tiles of one tensor: count tiles, the first at line line, each lines lines holding bytes row bytes but the last, last_lines holding last_bytes; the first’s first row row, each next row_step rows on.

Constants

ACT_BLOCKS_READ_LATENCY

Clocks from a lone block’s last beat into logos_act_blocks_p to its first beat’s read out of the ring: the beat’s maximum, the block’s, its scale, its place in the queue and the reader’s start.

ACT_CODE_LATENCY

Clocks from a value to its code in logos_act_code_p.

FOLD_LATENCY

Clocks from a fold lane’s operands to its element (logos_fold_lane_p): the function unit’s SiLU and the product after it, the sum held to them.

GREEDY_LATENCY

Clocks from a beat into logos_greedy to its token-row’s best: the four levels of its tree and the update.

STREAM_ENTRY_LINES

Lines of a streamed tensor’s StreamEntry.

Functions

act_blocks_out

The clock of each block’s out_v from logos_act_blocks_p, for blocks given in order as the clock of each one’s last beat and its beats: a block is read a beat a clock from ACT_BLOCKS_READ_LATENCY clocks after its last beat or from the clock after the block before it is read, whichever is later, and its out_v is the clock after its last beat’s codes land, ACT_CODE_LATENCY and the read’s clock after that beat’s read.

act_stream_p_verilog

The streaming engine’s activation quantizers with the code unit logos_act_code_p: logos_act_stream_p (logos_act_stream with its code units pipelined and its beats in registered-read block RAM), a block at a time, and logos_act_blocks_p, blocks back to back, each block’s codes out a beat a clock behind its scale; the reference quantizer’s modules beside them.

fold_lane_verilog

logos_fold_lane_p, a fold’s element from the element its slot held and the sum (Fold::apply).

greedy_verilog

logos_greedy, a greedy head’s reduction of a matvec job’s sums to each token-row’s first largest.

streaming_verilog

The streaming matvec of each named format set on channels of layout’s lines, with everything it is built from.

streaming_verilog_folding

streaming_verilog with each unit folding a job’s sums into its output slot over folds fold lanes, 1, 2, 4, 8 or 16 (fold_unit); 0 a unit that writes each sum as it is, streaming_verilog’s.

tile_run_verilog

logos_tile_run: a TileRun walked into tile descriptors, one a handshake, from start until its last is taken.