Module systolic 

systolic — the weight-stationary spatial GEMM tile (Phase 1).

Today’s GEMM lowering is a 1-D unrolled dot-product reduction (P multiply tiles → one accumulator → one output per operand pass): a MAC lane. A real accelerator (TPU MXU, a SambaNova PCU in GEMM mode) keeps the weights stationary in a 2-D array of PEs, streams activation rows through, and produces a whole output row in parallel — so each weight is read once and reused across the entire activation stream (the operand-reuse the reduction lane lacks), and N outputs settle per cycle instead of 1.

A Systolic tile is K rows × N cols of int8 multiply-accumulate PEs. PE(r, c) holds the stationary weight W[r][c]. Each cycle one activation row A[i][0..K] is applied and one output row O[i][0..N], with O[i][c] = Σ_r int8(A[i][r]) · int8(W[r][c]), is registered out. eval_systolic is the reference semantics; emit_systolic_verilog is the registered array; the per-PE datapath psum + a·w is SAT-certified ≡ its abstract form by the existing placer + bitblast (see the lock).

The datatype is int8 operands accumulated in wrapping i64 (the VM’s Int), so for int8-range inputs the tile is byte-identical to a plain integer GEMM — and to the VM. Operand truncation is x as i8 as i64, matched exactly by the RTL’s $signed(x[7:0]) sign-extension, so eval ≡ RTL for ALL inputs.

Structs

FusedBlock

A fused transformer-block stage over a weight-stationary GEMM tile (Phase 3): the GEMM, its fixed-point requantization (acc·M) >> S (arithmetic shift = floor-divide by 2^S), a per-column bias add, and a ReLU — all fused into ONE spatial pass so the intermediates are streamed, never materialized to memory.

Systolic

A weight-stationary GEMM tile: k contraction rows × n output columns of int8 MAC PEs, weights baked.

SystolicOs

An output-stationary tile: an r×c grid of accumulators computing O = A·W with contraction depth k (A is r×k row-major, W is k×c row-major, O is r×c row-major). Both operands stream in; O stays put.

Enums

SystolicSchedule

The systolic microarchitecture dial. Both schedules compute the identical GEMM (eval_systolic); they trade against each other rather than dominate, so both stay live (see the dominance characterization in pipelined_systolic_lock). SystolicSchedule::Combinational accumulates each column with a K-deep adder tree — one output row per clock, zero fill latency, but the critical path grows with K. Pipelined registers the activation (east) and partial sum (south) at every PE so the only path between flip-flops is one mul + one add — the critical path is K-INDEPENDENT (a real MXU stays at GHz at 128×128), at the cost of K+N−1 fill latency and the extra registers. Default is Combinational (smallest; correct for low clocks).

WeightSource

Where a PE array’s weights come from: baked into the netlist as constants, or loaded at run time through a row port (w_load, w_row, w_data = one packed row of n weights) — the spatial IR’s Compute(MatmulTiles) stage on our fabric loads its weight block, then streams activations.

WeightStart

What a loaded array’s weight registers hold before their rows are first loaded.

Constants

ACC_BITS_FULL

Default accumulator width — 64-bit, so emit_systolic_pipelined_verilog_bits is bit-identical to the i64 model for EVERY input. Real int8 MXUs (and the taped tiles) use a narrower ACC_BITS_MXU accumulator.

ACC_BITS_MXU

The TPU-MXU-standard int8 accumulator width (int32). Narrows the inter-PE partial-sum bus so a systolic grid routes without congestion; lossless for int8 with contraction depth K where |Σ a·w| < 2³¹.

Functions

bist_tile_weights

The stationary weights a signed-off weight-stationary tile bakes: a fixed int8 pattern, k*n row-major.

emit_loadable_vs_baked_testbench

A bench that loads sys’s weights into the loadable array while both arrays sit in reset, releases both on the same edge, streams acts (m rows) through both, and compares every output every cycle. Prints OUT o <i*n+c> <value> for the loadable array’s aligned output rows (row i sampled after edge i+L-1), LATENCY <L>, and PASS: iff the two arrays never differed.

emit_systolic_bist_wrapper

The self-test wrapper a weight-stationary tile core is signed off in: a 64-bit LFSR drives the k int8 activation lanes and the n registered outputs fold into a MISR, so only clk, rst and sig cross the block’s boundary and no pad shapes its timing.

emit_systolic_os_bist_wrapper

The self-test wrapper an output-stationary tile core is signed off in: the LFSR drives the r west activation and c north weight lanes, and the r × c stationary accumulators fold into the MISR through registers, row by row. Each row’s accumulators fold into a row word, the row words into two half-folds (foldA, foldB), and the MISR XORs those. The fold stays off the critical path, so the block’s critical path is the array’s PE, and no register reads more than one row’s accumulators, so no fold gathers every accumulator at one point of the die.

emit_systolic_os_pipelined_verilog

Emit the output-stationary pipelined array (int8) as synthesizable Verilog. See emit_systolic_os_pipelined_verilog_bits.

emit_systolic_os_pipelined_verilog_bits

Emit the output-stationary pipelined array at precision bits with a full (64-bit) accumulator.

emit_systolic_os_pipelined_verilog_bits_acc

Emit the register-staged output-stationary array at precision bits. Ports: clk/rst, aw{i} (west activation stream for row i), wn{j} (north weight stream for col j), and acc{i}_{j} (the stationary accumulator outputs, continuously readable). Per PE: a_reg (flows east), w_reg (flows south), acc_reg <= acc_reg + a_in·w_in. The only combinational path between flip-flops is one mul + one add — K-independent, exactly like the weight-stationary pipelined tile. Skew of the streamed operands is applied on-chip (row i / col j delayed i / j cycles), so the caller drives un-skewed streams.

emit_systolic_os_streaming_verilog

Emit the output-stationary array for tile after tile. Ports: clk/rst, first (high on each tile’s first contraction beat), aw{i} / wn{j} (the un-skewed operand streams of emit_systolic_os_pipelined_verilog_bits_acc) and res{i}_{j} (the finished sums of the previous tile). first rides the activation wavefront, so it reaches PE(i,j) with the tile’s first products: there the PE moves its sum into res_reg and restarts from that product — no reset and no idle beat between tiles. Column j’s result waits os_streaming_deskew cycles, so each row is readable all at once at os_streaming_result_offset.

emit_systolic_pipelined_loadable_verilog

A weight-stationary PE array whose weights are loaded through a port and hold what they power up with until then: pe_array_verilog(.., Loaded(PowerUp)). Cycle-for-cycle identical to the baked array once the same weights are loaded (locked).

emit_systolic_pipelined_loadable_verilog_starting

emit_systolic_pipelined_loadable_verilog with its weight registers starting at start.

emit_systolic_pipelined_verilog

Emit the PIPELINED weight-stationary array (int8) as synthesizable Verilog. See emit_systolic_pipelined_verilog_bits; the critical path is ONE mul + ONE add (K-independent).

emit_systolic_pipelined_verilog_bits

Emit the pipelined weight-stationary array at precision bits with a full (64-bit) accumulator.

emit_systolic_pipelined_verilog_bits_acc

Emit the pipelined, register-staged weight-stationary systolic array at operand precision bits and accumulator width acc_bits. Structure (matches eval_systolic_pipelined_acc register-for-register): per PE an activation register (a_reg, flows EAST, bits wide) and a partial-sum register (ps_reg, flows SOUTH, acc_bits wide) with ps_reg[r][c] <= ps_in + a_in·W[r][c] — the only combinational path between flip-flops is one bits×bits multiply + one add, so the critical path does NOT grow with K. On-chip input-skew (row r delayed r cycles) and output de-skew (col c delayed N-1-c) keep the clean clk/rst/a0..a{k-1}/o0..o{n-1} interface; o{c} on cycle t holds output row t-L (L=K+N-1). Narrowing acc_bits to ACC_BITS_MXU shrinks the inter-PE bus for a congestion-free tape and is lossless while the true accumulation fits acc_bits (guarded in the lock).

emit_systolic_scheduled

Emit the weight-stationary array under the chosen SystolicSchedule at precision bits. Both schedules are observationally equivalent (eval_systolic); this is the single retargeting seam an emitter/target selects the microarchitecture through.

emit_systolic_verilog

Emit the registered weight-stationary array as synthesizable Verilog. Ports: clk, rst, the k activation inputs a0..a{k-1} (64-bit; the low 8 bits are the int8 operand), and the n registered outputs o0..o{n-1}. Weights are baked as signed 64-bit constants (the sign-extended int8). One output row is produced per clock; o{c} on cycle t+1 is the dot of the activation row applied at cycle t.

emit_systolic_verilog_bits

Emit the weight-stationary array at a chosen integer precision bits (8/16/32/64) — the precision dial’s RTL (Phase 4). Operands are the low bits of each input, sign-extended; weights baked as sign-extended two’s-complement constants. Matches eval_systolic_bits for ALL inputs.

eval_fused_block

Evaluate the fused block: O[i][c] = ReLU( ((A·W)[i][c] · M) >> S + bias[c] ) — one streamed pass, no intermediate tensor written back. Byte-identical to computing the four stages separately (proven by the lock), so the fusion is correctness-preserving; the win is that nothing lands in DRAM between stages.

eval_systolic

Reference semantics: O(M×N) = A(M×K) · W(K×N), int8 operands accumulated in wrapping i64. acts is m*k row-major (A[i][r] = acts[i*k + r]); returns O as m*n row-major.

eval_systolic_bf16

bf16 GEMM: operands rounded to bf16, products and accumulation in f32 (the standard bf16 MAC), returned as f64. weights is k*n row-major, acts is m*k row-major.

eval_systolic_bits

Precision-parameterized integer semantics: operands truncated to bits (8/16/32/64), products accumulated in wrapping i64 — the integer precision dial (Phase 4).

eval_systolic_f32

fp32 GEMM: operands, products and accumulation in f32 (the PCU’s native 32-bit FU MAC — Plasticine’s format), returned as f64. weights is k*n row-major, acts is m*k row-major.

eval_systolic_os

Reference int8 GEMM for the output-stationary tile: O[i][j] = Σ_t A[i][t]·W[t][j] (wrapping i64).

eval_systolic_os_pipelined

Cycle-accurate output-stationary model — mirrors the RTL register-for-register. Activation A[i][t] enters row i skewed by i, flows east; weight W[t][j] enters col j skewed by j, flows south; every cycle each PE does acc <= acc + a_in·w_in (skew/drain fill zeros contribute nothing). At PE(i,j) on cycle T both operands carry contraction index t = T-i-j, so acc[i][j] converges to O[i][j]. Returns (O, latency) with latency = pipelined_os_latency.

eval_systolic_os_pipelined_acc

eval_systolic_os_pipelined with an explicit accumulator width acc_bits — each PE’s accumulator wraps (two’s-complement) at acc_bits, exactly like the acc_bits-wide acc_reg in the emitted OS RTL.

eval_systolic_os_pipelined_o

eval_systolic_os_pipelined without the latency — the O matrix only.

eval_systolic_pipelined

Cycle-accurate golden model of the pipelined weight-stationary array — steps the register planes exactly as the emitted RTL does (activation EAST, partial-sum SOUTH, on-chip skew/de-skew), so the RTL must match it cycle-for-cycle. Returns (O row-major, L) where O ≡ eval_systolic_bits (the math is unchanged; only the timing differs). acts is m*k row-major.

eval_systolic_pipelined_acc

eval_systolic_pipelined with an explicit accumulator width acc_bits — the partial-sum register wraps (two’s-complement) at acc_bits, exactly like the acc_bits-wide ps_reg in the emitted RTL. With acc_bits = 64 this is bit-identical to eval_systolic_pipelined; with ACC_BITS_MXU it models the int32-accumulator tape and is lossless whenever the true accumulation fits (guarded in the lock).

eval_systolic_pipelined_o

eval_systolic_pipelined without the latency — the O matrix only, for equivalence tests.

eval_tiled_gemm

Compute a K×N GEMM by TILING into tk×tn sub-tiles (Phase 2). Each weight sub-block is loaded ONCE into the on-chip scratchpad and reused across all m activation rows; k-direction tiles are accumulated. Byte-identical to eval_systolic on the full matrix — so the tiling is proven correct by equality to the untiled reference (which is itself ≡ the oracle ≡ VM). acts is m*k row-major.

eval_unfused_block

The UNFUSED reference: the same computation with each stage materialized into its own buffer (a DRAM round-trip per stage). Equal to eval_fused_block — the composition proof — but it writes 2·m·n intermediate elements the fused pass never does. Returns (result, intermediate_writes).

os_streaming_deskew

The cycles column j’s results wait so a whole row is readable in one cycle at os_streaming_result_offset: c - k - j where that is positive. A tile’s period is at least its contraction k, and column j holds its result for a period from period + i + j + 1, so it covers the row’s read cycle with this delay at every period a tile can have; an array no wider than its contraction waits for nothing.

os_streaming_result_offset

When row i’s results of a streamed tile are readable: period cycles after the tile’s first beat the next tile’s first beat arrives and moves each PE’s finished sum into its result register as it passes, i + j cycles in, where it stays period cycles. Every column of row i holds it period + i + c cycles after the tile’s first beat: column j from period + i + j + 1 until 2·period + i + j, delayed where that window would close before column c - 1’s opens (os_streaming_deskew).

pipelined_latency

The fill latency of the pipelined array: row i’s output emerges at cycle i + L, L = K + N − 1. Shared by the golden model, the RTL emitter, and the lock so they cannot disagree.

pipelined_os_latency

The cycle at which the last accumulator (i=r-1, j=c-1) has absorbed its final product and is drainable.

round_bf16

Round x to bf16 (8-bit exponent, 7-bit mantissa) with round-to-nearest-even — the float precision dial (Phase 4). Idempotent, and a no-op on values already representable in bf16.

trunc_signed

Sign-extend x from its low bits (the int-precision truncation; bits >= 64 is a no-op). Matches the RTL’s $signed(x[bits-1:0]).