Expand description
An int8 transformer as a LOGOS decode step: one token in, its logits and the next token out, the KV cache carried.
The model is integer arithmetic throughout — int8 weights, a requant (acc · M) >> S with int8 saturation between
layers, an exp-table softmax whose context is a floored weighted mean, a ReLU FFN, a tied head and an arg-max — so
the step is written in the integer tensor sugar (X times M shifted right by S, X clamped between LO and HI,
relu of X, lut attention of …) beside the shared forms (row tok of E, matmul, plus, Set row,
argmax of). The parser desugars every one into ordinary loops, so the VM’s run of the step IS its
meaning, and the spatial compiler lowers the same text onto the integer fabric.
Structs§
- Int8
Model - A one-layer int8 transformer: its shapes, its requant pairs, its tables.