Module passes 

The target-parametric passes: every one is fn(&SpatialProgram, &SpatialTarget) -> Result<SpatialProgram, SpatialReject>, reads only the program and the descriptor, and leaves validate() holding. The order is pinned in super::pipeline::PASS_ORDER. There is no strategy input anywhere: tile chooses the plan by the cost model, fuse_readers re-derives the same comparison, and every later pass is a function of the structure.

Re-exports

pub use crate::tensor_graph::front as recognize_stages;

Modules

compose

compose — N verified sections become ONE SpatialProgram: the persistent-decoder form.

forward

forward — one section’s result handed to the next section’s compute, never through memory.

from_q8

Q8_0 weights on the chip — a product y = x · W of up to a tile’s rows over a weight the deployment stores Q8_0.

from_qwen_ops

from_qwen_ops — every op of an authored transformer forward, lowered as a COMPOSITION of the closed compute algebra: chains of ComputeOp::Generic and ComputeOp::MatmulTiles, and nothing else.

from_tensor_op

from_tensor_op — a recognized tensor op becomes a TEMPLATE spatial program (phase Template): the tensors, the four stages, three double-buffered FIFOs, and one work item per output tile. The numerics are chosen HERE from the target’s matrix engine and the recognized fills, and the exactness claim is derived, never assumed: an Int source on a float-primary engine rides as bf16 operands with an fp32 accumulator and is claimed exact only while every partial sum fits fp32’s mantissa.

fuse_generic

fuse_generic — put back into one register window what legalization had to take apart.

fuse_readers

fuse_readers — reader fusion / multicast as a GENERAL pass, decided by the cost model.

pack

Phase packing — independent sections run at once, on disjoint sites.

persist

persist — one resident worker per site loops over ITS work items (the “persistent kernel”).

place

place — pin every work item, stage and buffer to a site and an engine.

section_shape

The canonical SHAPE of a section — its choreography with every name forgotten.

shard

════════════════════════════════════════════════════════════════════════════════════════════════ TENSOR PARALLELISM — one program over many RDUs, and the reduction that makes it mean something.

size_buffers

size_buffers — buffering as a GENERAL pass: depths from a rate model under the local-memory budget, a deterministic non-overlapping layout, typed refusals.

synthesize_handshakes

synthesize_handshakes — semaphore choreography DERIVED from the buffer/edge graph.

tile

tile — choose the per-site block plan from the legal lattice by the cost model.

verify

verify — attach the choreography certificate, or refuse.

Enums

SpatialReject

Why a program cannot become (or stay) a spatial program on a target — typed, never a silent fallback.

Constants

LV_GEN_COL

LV_GEN_K

A ComputeOp::Generic’s CONTRACTION axis: the third iteration variable, summed at every (row, col) point — C[r, c] = Σ_k A[r, k]·B[k, c] — so a small matrix product is one stage. Maps step along it; the output never does.

LV_GEN_ROW

A ComputeOp::Generic’s own iteration space: the rows inside a tile, and the columns across the operand’s tiles. Distinct from the choreography’s loop variables above, which index WORK; these index the ELEMENTS one compute stage walks, and are the axes an access map may omit to broadcast.

LV_ITEM

LV_ITEM_X

LV_ITEM_Y

The loop variables the choreography addresses tiles with.

LV_KBLOCK

LV_SUB_H

LV_SUB_W