Module emit 

The lane’s forms printed as one CUDA program, and the sidecar of values it reads.

A lane is the decode step and its prefill chunks (the same layers over a block of 2, 4, … rows), compiled as separate forms that share the model’s constants and its carried state. The program is run as <program> <checkpoint.gguf> <sidecar> <n_gen> [--logits <file>] <token> …. It maps the checkpoint (refusing a file other than the one it was compiled against: the size and the digest of everything before the data section are compiled in), uploads each quantized matrix’s rows straight from it, uploads the computed constants from the sidecar, and zeroes the carried state. Each form’s step is captured once as a CUDA graph (the cuda_graphs knob). The prompt runs as the largest chunks that fit, then one decode step per generated token: the token and the position live on the device, and each step’s arg-max moves both on, so generation needs no round trip to the host. The program prints the generated token ids, the prefill time and the decode rate; --logits also writes every generated position’s logits as little-endian f32.

Structs

Checkpoint

The checkpoint a program is compiled against.

Program

A compiled lane: the CUDA program and the sidecar it reads its computed constants from.

Functions

emit

Print forms as one CUDA program over the checkpoint ckpt, its caches ctx positions long.