Skip to main content

dense_tables

Function dense_tables 

Source
pub fn dense_tables(
    g: &TensorGraph,
    linked: &[usize],
    target: &SpatialTarget,
    agreed: &BTreeSet<String>,
) -> BTreeSet<String>
Expand description

The tables the ops linked of g lay out DENSE, with agreed, the tables a deployment’s other programs lay out dense: every K and V an attention reads in the flash form (and a block of rows’ queries and result, which the flash form reads and writes a tile of rows at a time), and every multi-row tensor an op reads or writes as a matrix (a stacked batch through a matmul, a pointwise op over a matrix). The layout is decided here, once per table, so a row write or a row read on a table and the ops that read it whole can never disagree on it.

★ A ROW NORM OVER A MATRIX HAS NO LAYOUT OF ITS OWN. It reads and writes in one layout, so it takes its tensors’ — the matrix when any of them is dense, a stack a matmul reads — and ties the others to it; left to itself it is the flat row every per-head norm is. The tie runs to a fixed point, since a norm’s input may be another norm’s output.