Module lane_shim 

What the compiled generation lane runs whole: the CUDA lane’s answer to the shared optimizer.

The shared rules decide a fusion by dataflow; the lane’s kernels decide whether a fused op has a launch. Every condition here is a fact about a kernel of the library the lane links:

• a per-head product reads a part of every unit of its input in place only at whole 32-value blocks, the quantized form’s; • an attention reads its values in place only from an even place at an even pitch, a pair at a time; • a convolution’s epilogue takes a per-head L2 norm over heads of at most 1024 values, whole warps of them, that divide its channels; • a gated pair of products is one launch as GEMVs (a decode step’s rows, a batch’s, or the precise numerics), or on the tensor cores as one GEMM of the gate’s and up’s tiles side by side where their formats agree and the hidden width is whole pair tiles; • a mixture folds its shared expert into a block of at most 32 warps of kept experts, and never writes its result at a row a parameter names; • a product by checkpoint rows reads its activation quantized under the lane’s quantized numerics — one int8 word a value, or two — and a producer quantizes in its epilogue under the fuse_quant knob where its kernel can: a norm, an elementwise op, a SwiGLU, a delta rule, a gated pair whose hidden width is whole 32-value blocks, a product of four rows a warp whose outputs are whole blocks; • several products of one activation are one tensor-core GEMM in parts — at most four, one format, each a whole number of 128-output tiles — above the GEMV’s rows.

The lane’s own rules link into the table: a wide chunk’s latent attention taken whole before the shared rules (the lane runs it expanded), and the decode step’s products of one activation as one GEMV in parts before the products on the tensor cores are merged.

Structs

LaneShim

The lane under its knobs, compiling a form whose products are a batch’s (each row its own sequence) or not, with the lane rules held back.

Latent

A latent attention’s shape and names, as its expanded launches read them: the query projection qr (each head’s unrotated part its first nope values) and the rotated part qpp; the cache kc, kw wide a row — the latent lora values then the shared rotated key — len rows; the key and value halves of the latent’s expansion and their formats (kro: the key half’s checkpoint rows are its outputs); the result ct.

Functions

chunks_apart

Whether an attention cut into splits chunks runs a block a chunk: where its rows’ head blocks alone leave the chip idle.