Module knobs 

The CUDA backend’s optimization knob lattice.

Design principle (the discipline this whole tier is held to):

Every optimization is a trade-off. If a transform is strictly better — Pareto-dominant, i.e. faster (or smaller) with no downside for any valid program — it is baked in UNCONDITIONALLY and gets no knob (see STRICTLY_BETTER). A knob exists only where the transform wins in some regimes and loses in others. Each knob then defaults to the best general case, and records its trade-off (CudaKnobs::tradeoffs) so an autotuner — or a human — can search the lattice with eyes open, never blindly.

This mirrors the machinery the codebase already uses: crate::codegen_sva::fabric_target::FabricTarget (a capability lattice), spatial::passes::tile::best_tile_config (a cost-model search over a TileConfig lattice), and the published Pareto frontier. The knobs here are the same idea aimed at the CUDA emitters: the GEMV/GEMM kernels and (as they land) the transformer orchestration (KV-cache, CUDA-graph capture, quant fusion). The set of legal knob assignments is a lattice; CudaKnobs::default is its best-general point and every other point is reachable by flipping a documented trade-off.

Structs

CudaKnobs

The CUDA backend’s optimization knobs — one point in the lattice. Construct with CudaKnobs::default for the best-general point and flip individual knobs to trade off. Every field is a trade-off; there are deliberately no fields for strictly-better transforms (those are always applied, see STRICTLY_BETTER).

KnobTradeoff

A single knob’s trade-off — why it is a knob and not a baked-in transform.

Enums

CachePrecision

The precision an attention’s cache is stored in (CudaKnobs::cache_precision).

LoadWidth

GEMV/GEMM weight-load width. Wider loads expose more memory-level parallelism (the Q8_0 forward is weight-bandwidth-bound), but cost registers per thread.

MatmulPrecision

Contraction precision for a lowered matmul.

ResolvedPrecision

What a MatmulPrecision actually lowers to after Auto is resolved against what we can prove.

StatePrecision

The precision a recurrent memory is stored in between steps (CudaKnobs::recurrent_state).

Constants

STRICTLY_BETTER

Transforms that are strictly better — Pareto-dominant, never worse for any valid program — and are therefore applied UNCONDITIONALLY with no knob. If any entry here ever acquires a downside, it must grow a CudaKnobs field and move out of this list (the knob-lattice lock enforces the disjointness).