Skip to main content

Module vectorize

Module vectorize 

Source
Expand description

Element-wise map vectorization — the general, sound SIMD lever.

Recognizes a region that is a PURE ELEMENT-WISE MAP — a loop for i in 1..=n (1-based, the Logos array convention) whose body reads one or more arrays at index i, computes a straight-line float expression, and writes the result to an array at index i, with the induction variable i as the ONLY loop-carried value. Such a loop is bit-exactly vectorizable: lane i computes exactly f(a[i], b[i], ...) whether scalar or packed, so 2-wide (or wider) SIMD changes nothing about the per-element result — unlike a REDUCTION (sum += a[i]), whose 2-lane form reassociates the float adds and is therefore NOT bit-identical (and is rejected here).

This is the recognizer half (a pure analysis). The codegen half lowers each body op to its packed form (AddF->addpd, …) over 2 lanes with a scalar tail; the packed primitives live in crate::x64asm.

Structs§

MapPlan
A recognized element-wise map loop. The straight-line load/compute/store body is ops[body_start..body_end]; the guard, increment, and back-edge surround it.

Functions§

emit_map_kernel
Emit a complete vectorized loop kernel for a recognized element-wise map, frame-ABI extern "C" fn(*mut i64) -> i64: it reads the induction i, limit n, and each array base pointer from their frame slots ([rdi + slot*8]), runs a 2-wide packed loop (i += 2) over full pairs, then a single-element scalar tail that REUSES the packed body (load one element with movsd so lane 1 is zero, run the same packed ops, store lane 0 with movsd — the junk lane is never stored and masked SSE faults are harmless), writes i = n back, and returns 0. Returns None (caller falls back to the scalar region) if the body needs more than 4 distinct arrays or 14 lane temps, or uses an op the kernel does not lower (e.g. LoadConst).
emit_packed_arith
Lower one straight-line float-arithmetic body op to its 2-wide PACKED form, computing dst = lhs OP rhs over both lanes. xmm(slot) maps a body slot to its assigned lane register; scratch is a free XMM for the awkward dst == rhs non-commutative case. SSE packed ops are 2-operand (x OP= y), so this materializes the 3-operand dst = lhs OP rhs form, never clobbering an operand it still needs. Loads/stores/LoadConst are handled by the loop driver, not here.
recognize_elementwise_map
Returns Some(MapPlan) iff ops is a pure element-wise map region in the top-tested while shape that adapt_region produces: