pub fn vendor_dispatch_blocking(
spec: &MatmulSpec,
cfg: &TileConfig,
) -> (usize, usize, usize)Expand description
Emit a vendor-dispatch program: the compiler recognized the matmul, so it routes the primitive to the
target’s optimal op — ttnn.matmul on Tenstorrent — the same way a C compiler recognizes a matmul loop
and calls BLAS. It passes the compiler’s OWN auto-tuned config (grid, per-core, sub-block, inner block),
so the dispatched program runs at the compiler’s tuning on the vendor’s optimized implementation. For
640³ this measured 6154 device cycles — beating ttnn’s own default (6745). Generated from (spec, cfg).
The secondary blocking the vendor dispatch runs with, for a recognized shape and the autotuner’s tiling:
(in0_block_w, out_subblock_h, out_subblock_w). ONE definition, used by the dispatch emitter and by the
measurement harness’s tiling printer, so “our config” can never mean two different things.
The inner block is the largest divisor of Kt that is at most 4 (measured: bigger inner blocks lose — at 2048x1024x512, in0_block_w 8 costs 876 cycles over 4 and 16 costs 2,629). The sub-block is NARROW, one row tall: measured on both shapes swept, 1x2 beats the square 2x2 by 368 cycles at 640^3 and by 647 at 2048x1024x512, where it also beats ttnn’s own heuristic. Smaller sub-blocks buy finer pack/math overlap and a smaller register reload at the last K block, which outweighs the extra acquires.