Module shard 

════════════════════════════════════════════════════════════════════════════════════════════════ TENSOR PARALLELISM — one program over many RDUs, and the reduction that makes it mean something.

An SN50 reaches its token throughput by splitting one model across chips; the published 16-chip tensor-parallel run is the shape this pass exists to express. Before it, our tensor parallelism was arithmetic in spatial/fit.rs — weight bytes DIVIDED by a chip count and reported to fit. That says a sharded model would fit. It does not say what a sharded model COMPUTES.

★ WHAT A ROW-PARALLEL SHARD ACTUALLY IS. For C(m×n) = A(m×k)·B(k×n), split the CONTRACTION axis k into chips slices. Chip c holds A[:, kc] and B[kc, :] and computes

C_c = A[:, kc] · B[kc, :]

which has the FULL m×n shape and is a PARTIAL SUM — it is missing every term outside its slice. The answer is C = Σ_c C_c, and that sum is the all-reduce. This is the whole hazard: C_c is correctly shaped, fully written, and wrong. A shard that never reduces returns Ok with 1/chips of the answer, so the reduction is not an optimization to be scheduled later — it is part of the meaning of the program, and crate::spatial::certificate carries an obligation that says so.

★ THE CHIP IS A COORDINATE, NOT A COMPILATION. Sharding does not produce chips programs. It produces ONE program whose sites carry a chip index, because the all-reduce is an edge that spans chips and there is no single-chip program in which that edge can be written down. Everything downstream — placement, buffer sizing, handshake synthesis, the certificate — then sees the collective the way it sees every other hand-off.

★ THE COLLECTIVE’S RECEIVE SIDE IS STILL MISSING, and the IR is right to refuse it. validate_ports records the rule: “inter-site edges name the SOURCE buffer at the sender; the receivers own their own copies.” A cross-site edge is therefore NOT a producer of the sender’s buffer — so a reduce that reads the per-chip partial buffers directly leaves each of them with zero producers (FifoProducerCount { buffer, count: 0 }). That is not a technicality: it is the IR insisting that data crossing a chip boundary LANDS SOMEWHERE, in a buffer the receiving site owns. StageKind::Recv is already exactly this shape for multicast, and the collective must follow it — one receive per chip at the reducing site, the fold reading those local copies. Until that lands, shard builds a program the validator refuses, which is the honest state: it never returns a shard it cannot stand behind.

★ WHY THIS RUNS ON A PLACED PROGRAM. place and tile are no-ops once their results exist, so sharding a verified program and re-running the pinned passes rebuilds exactly the parts that must change (buffers, handshakes, schedule, certificate) and preserves the tiling decision the cost model already made. The chip assignment this pass writes therefore survives, which it would not if the program were re-placed from scratch. ════════════════════════════════════════════════════════════════════════════════════════════════

Structs

ShardPlan

How a program is split across chips.

Enums

ShardError

Functions

shard

Split p over plan.chips RDUs along the contraction axis, insert the all-reduce, and re-run the pinned passes so the collective is scheduled and certified like any other hand-off.