Struct Cuda 

pub struct Cuda;

The CUDA lane (crate::codegen_cuda::lane): the decode step and its prefill chunks as one fused CUDA program and the sidecar of computed constants it reads.

Trait Implementations

impl Backend for Cuda

fn run( &self, _spec: &ModelSpec, checkpoint: &Path, ctx: usize, out: &Path, sequences: &[Sequence], ) -> Result<Generated, Refused>

The lane built by nvcc for this machine’s GPU once and run on the checkpoint a sequence a run: its own clocks time each prompt’s prefill and every decode step after its first token.

fn name(&self) -> &'static str

The chip, as the command line names it.

fn bring_up( &self, spec: &ModelSpec, checkpoint: &Path, ctx: usize, out: &Path, ) -> Result<Brought, Refused>

Lower spec’s program — the checkpoint at checkpoint — over caches of ctx positions, writing the artifacts under out.