Skip to main content

from_rope

Function from_rope 

Source
pub fn from_rope(
    spec: &RopeSpec,
    names: &StageNames,
    target: &SpatialTarget,
) -> Result<SpatialProgram, SpatialReject>
Expand description

RoPE-NEOX over h heads of hd dims, rotating each head’s leading 2·half dims as (i, i+half) pairs and passing the remaining hd − 2·half through — Qwen3.8’s partial rotary.

★ THREE REGIONS PER HEAD, ALL BANDS OF ONE FLAT AXIS. Head h’s rotated first half is the columns [h·hd, h·hd + half), its second half [h·hd + half, h·hd + 2·half) and its pass-through tail [h·hd + 2·half, (h+1)·hd). Each bound is a CONSTANT, so the whole op is one Generic with 3·h − 1 predicates and no per-head stages at all. q is read through three shifted maps (c, c + half, c − half) shared by every head; the cosine/sine tables are read through a per-head base, because their index is the position WITHIN the head.

★ A CACHE IS READ WHERE THE POSITION POINTS, IN THE ROPE’S OWN SECTION. With from position P the tables are cache rows of angles and row r of the block turns by row P + r. Rows of WHOLE TILES are fetched alone: the block’s rows·half angles are a tile run from tile (P − 1)·half_t — a literal in the load’s base, a named P an IndirectTile over the guarded P − 1 — so a real model’s cache of thousands of positions never comes on chip. Rows narrower than a tile have no tile of their own: the cache is loaded whole and every table read adds the flat offset (P − 1)·half — a literal as one more constant, a named one as an IndirectTerm checked against the positions the block admits. Either way a position past the cache faults where every tier’s check fails instead of reading angles that are not there.