Skip to main content

compile_program_to_chips

Function compile_program_to_chips 

Source
pub fn compile_program_to_chips(
    source: &str,
    target: &SpatialTarget,
    manifest: &TpManifest,
) -> Result<ChipLowering, SpatialReject>
Expand description

★ A CHIP BOUNDARY IS A TRANSFER, NOT A SHARED MEMORY. compile_program_to_spatial_tp lowers a tensor-parallel program as ONE program over its chips, every section followed by a barrier the chips all join — the shape of a machine whose chips share one fabric. Boards joined by a link share nothing: each runs its own program, and a tensor one board writes and the other reads crosses the link between them. So each chip’s statements are lowered alone, in order, as an ordinary one-chip program, every tensor narrowed by the interval the WHOLE program proves for it, and what crosses is listed (super::tensor_parallel::transfers). An all-reduce is then what it is over a link: the partials cross to the chip that folds them, and the fold is that chip’s own addition.