Struct LaneShim 

pub struct LaneShim {
    pub knobs: CudaKnobs,
    pub batch: bool,
    pub held: Vec<&'static str>,
}

The lane under its knobs, compiling a form whose products are a batch’s (each row its own sequence) or not, with the lane rules held back.

Fields

knobs: CudaKnobs

batch: bool

held: Vec<&'static str>

Implementations

impl LaneShim

pub fn per_warp(&self, p: &GemvForm) -> usize

The output rows a warp of a product’s kernel takes: four for a wide product in a block format met one row at a time (rows_a_warp), one otherwise.

pub fn on_tensor_cores(&self, p: &GemvForm) -> bool

Whether a product multiplies on the int8 tensor cores: past the GEMV’s rows, not a batch’s, under the quantized numerics, by checkpoint rows.

impl LaneShim

pub fn attention(activations: Q8MatKind) -> Option<AttentionNumerics>

How a wide chunk’s attention takes its operands at activations: on the f16 tensor cores under one word (its queries, keys, values and weights rounded to f16, lk_attention_mma); on the int8 tensor cores under two (lk_attention_i8: each query, key and value row in two words under its largest over 127, each weight times its value row’s scale in three under the largest of its row in the tile, every product of the words taken); in f32 under the dequantized numerics.

pub fn latent_launches(&self, l: &Latent) -> Option<[Launch; 3]>

A latent attention’s expanded launches: the key half and the value half over every cached latent row, then the attention of each head’s split query over its own keys and values — None where one block’s shared memory holds no head.

Trait Implementations

impl Clone for LaneShim

fn clone(&self) -> LaneShim

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LaneShim

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Shim for LaneShim

fn name(&self) -> String

The chip and the configuration it answers for, e.g. cuda-lane:rtx3060:TwoPassInt8.

fn supports(&self, class: ClassId) -> Support

What the chip’s link does with class. Every chip answers by a match over every class with no catch-all arm, so a class added to the registry fails the build until each chip answers it.

fn fits(&self, op: &Op, g: &TensorGraph) -> Result<(), String>

Whether the chip’s link builds op as it stands — its shapes, its accesses, its destination: Ok, or why not.

fn activation( &self, consumer: &Op, operand: usize, g: &TensorGraph, ) -> Option<QuantFormat>

The quantized form consumer reads its operand operand in, or None for one it reads as its values.

fn rules(&self) -> Vec<(Position, ChipRule<'_>)>

The chip’s own rules, each linked into the table where its position says.

fn numerics(&self) -> Option<Contract>

The numerics contract of the knob point the chip answers for, or None while the chip has not declared it.