Module compile

Expand description

LOGOS Compilation Pipeline

This module provides the end-to-end compilation pipeline that transforms LOGOS source code into executable Rust programs.

§Pipeline Overview

LOGOS Source (.md)
      │
      ▼
┌───────────────────┐
│  1. Lexer         │ Tokenize source
└─────────┬─────────┘
          ▼
┌───────────────────┐
│  2. Discovery     │ Type & policy definitions
└─────────┬─────────┘
          ▼
┌───────────────────┐
│  3. Parser        │ Build AST
└─────────┬─────────┘
          ▼
┌───────────────────┐
│  4. Analysis      │ Escape, ownership, verification
└─────────┬─────────┘
          ▼
┌───────────────────┐
│  5. CodeGen       │ Emit Rust source
└─────────┬─────────┘
          ▼
    Rust Source

§Compilation Functions

Function	Analysis	Use Case
`compile_to_rust`	Escape only	Basic compilation
`compile_to_rust_checked`	Escape + Ownership	Use with `--check` flag
`compile_to_rust_verified`	All + Z3	Formal verification (requires `verification` feature)
`compile_project`	Multi-file	Projects with imports
`compile_and_run`	Full + Execute	Development workflow

§Examples

§Basic Compilation

let source = "## Main\nLet x be 5.\nShow x.";
let rust_code = compile_to_rust(source)?;
// rust_code contains:
// fn main() {
//     let x = 5;
//     println!("{}", x);
// }

§With Ownership Checking

let source = "## Main\nLet x be 5.\nGive x to y.\nShow x.";
let result = compile_to_rust_checked(source);
// Returns Err: "x has already been given away"

Re-exports§

pub use crate::concurrency::marshal::decode_value_raw;
pub use crate::concurrency::marshal::encode_value_raw;

Structs§

AotModule: A compiled AOT-native module (HOTSWAP §Axis-3): the cdylib Rust SOURCE plus the exact exported symbol to resolve and its arity — everything the loader needs.
CheckArtifacts: Compile LOGOS source and write output to a directory as a Cargo project.
CompileOutput: Full compilation output including generated Rust code and extracted dependencies.
CrateDependency: A declared external crate dependency from a ## Requires block.
GenuineProjectionResult: Result of a genuine Futamura projection via self-application. Contains the LOGOS source of the residual and the discovered entry points.
RunOutcome: What a program run produced: every output line emitted before completion or failure, plus the error if it failed. The differential contract is vm_outcome(src) == tw_outcome(src) — output AND error.

Enums§

CompileError: Errors that can occur during the LOGOS compilation pipeline.

Functions§

aot_build_function: On-demand AOT-native build + load (HOTSWAP §Axis-3): compile fn_name (and its callees) to an optimized cdylib (cached), dlopen it, and return the loaded crate::vm::NativeFn ready to install via Vm::install_aot_native. None if the function is absent / outside the scalar subset / the build or load failed — the caller keeps it on VM+JIT. Blocking (runs rustc); the background worker (BgAotCompiler) calls this off the interpreter thread.
aot_cache_key: The cache key for an AOT-native artifact (HOTSWAP §Axis-3 / P16): a deterministic FNV-1a hash of the rustc toolchain version AND the optimized Rust source. The toolchain is in the key because Rust has no stable cross-version ABI — a different rustc/runtime yields a different key, so a stale, ABI-mismatched .so is NEVER reused (soundness over convenience). The source captures the function, its callees, and the optimization config (it IS the codegen output).
aot_load_bundle: Load every native-annotated function in source as a compiled-native crate::vm::NativeFn (HOTSWAP §Axis-3): the run path installs these via Vm::install_aot_native, so an annotated function dispatches to rustc -O3 machine code from its first call. Uses the persistent cache, so a bundle pre-built by largo build --native-functions is a cache hit — no rustc on the run path. A function outside the scalar subset / failing to build is skipped (stays on VM+JIT).
build_native_bundle: Pre-build every native-annotated function in source into the AOT bundle under bundle_dir (HOTSWAP §Axis-3 / P16 — the “tools to bundle them”). Returns the (function_name, cdylib_path) manifest. Functions outside the sound scalar subset are silently skipped — they keep running on VM+JIT, no gap at the seam. Err only on a genuine parse/type error.
build_native_cdylib: Build AOT-native cdylib Rust source (from compile_function_to_native_rust) into a loadable cdylib (HOTSWAP §Axis-3 / P16). Generates a minimal crate under work_dir/<crate_name>, copies the shared runtime crates (so the .so and the interpreter share the SAME logicaffeine_data ABI), and runs cargo build --release. Returns the path to the produced dynamic library. panic = "unwind" (the default) — NOT abort — so a panic in a loaded function can be contained rather than killing the host process.
build_native_cdylib_cached: build_native_cdylib with a persistent cache (HOTSWAP §Axis-3 / P16). The artifact lives at cache_dir/aot_<fn>_<key>/… keyed by aot_cache_key; an identical function reuses its .so across runs (no rebuild), and a toolchain change yields a new key so the stale library is never loaded.
build_native_pe_binary
build_native_pe_cdylib: Build (and cache) the PE compiled as an IN-PROCESS cdylib exporting logos_pe_specialize. Mirrors build_native_pe_binary but emits a cdylib whose lib.rs is the generated PE engine (pe_cdylib_source) plus a hand-written extern "C" shim ([PE_CDYLIB_SHIM]) that decodes the wire CProgram and calls peSpecializeOnce directly — so the caller loads it once and specializes with a plain function call, no process, no pipe. Content-addressed on the (stable) cdylib PE source + shim + toolchain, so warm calls skip rustc.
build_native_wasm: Build a function’s AOT module to a browser-loadable wasm32-unknown-unknown cdylib (HOTSWAP §Axis-3 / P17 — the browser analog of build_native_cdylib). The module is the same scalar-ABI shim; the wasm crate keeps logicaffeine-data + logicaffeine-system (both wasm-ready) but drops the desktop-only tokio, the logicaffeine-system full feature, and target-cpu=native. Returns the .wasm path; the browser then WebAssembly.instantiates it and the entry calls into it through the warm-bytecode indirection. Requires the wasm32-unknown-unknown target.
classify_source: Phase 0 (work/FINISH_INTERPRETER.md): classify the determinacy of a LOGOS program.
compile_and_run: Compile and run a LOGOS program end-to-end.
compile_file: Compile a LOGOS source file. For single-file compilation without dependencies.
compile_function_to_native_rust: Generate the Rust SOURCE for an AOT-native cdylib of ONE function (HOTSWAP §Axis-3 / P14b): the target function plus its transitive callees (crate::codegen::function_slice), emitted through the normal ARCHITECT codegen, with the crate::codegen::codegen_native_tier_export shim appended so the loader can resolve the logos_native_<target> symbol. Mirrors compile_program_full’s front-end, then slices before codegen.
compile_program_full
compile_program_full_deterministic: Compile LOGOS source and return full output including dependency metadata.
compile_program_full_with_proven: Like compile_program_full, but bundles an extracted math/logic module (proven) into the generated Rust — the imperative half of a mixed document. The module is emitted as pub mod proven { … } use proven::*; so the imperative program can call its functions / check_* predicates by name. None (or a blank module) yields output byte-identical to compile_program_full.
compile_program_full_with_wire: Like compile_program_full, but every emitted (non-generic) enum also gets WireEncode/WireDecode impls over the shared logicaffeine_data::wire codec — byte-identical to the peer codec’s encode_value_raw. This lets a compile-once native partial evaluator receive a program AST as data over the same fast codec the interpreter uses. Ordinary compiles (the default) are byte-unchanged.
compile_program_traced: Compile source while recording which optimizations actually FIRED — the honest, single-compile alternative to optimizations_used’s O(N) diff. An optimization is “fired” when it actually changed the program or emitted its optimized form for this compile, under the effective (normalized) config. Reusable: any caller can compile a program and learn what fired, for tracing and tooling.
compile_project: Compile a multi-file LOGOS project with dependency resolution.
compile_to_c: Compile LOGOS source to C code (benchmark-only subset).
compile_to_dir
compile_to_rust: Compile LOGOS source to Rust source code.
compile_to_rust_checked: Compile LOGOS source to Rust with ownership checking enabled.
compile_to_rust_deterministic: Compile in Mode B (deterministic replay): identical to compile_to_rust except a Select lowers to the seeded winner-pick that shares the interpreter’s choice function, so a binary run under LOGOS_SEED=… reproduces the interpreter’s selection. Used by largo build --deterministic and the seeded differential harness. Non-Select emission is unchanged.
compile_to_rust_traced: One compile returning the generated Rust, the fired-optimization keywords, AND the (winner, loser) precedence decisions made — what an interactive viewer calls per toggle state to show the firing opts and the conflicts that occurred.
compile_to_rust_with_proven: Compile imperative LOGOS to Rust with an extracted math/logic module bundled in. proven is a main-less Rust module body (the Forge’s extraction); it is emitted as pub mod proven { … } use proven::*; so the imperative program can call its functions / check_* predicates by name. See compile_program_full_with_proven.
compile_to_wasm: Compile LOGOS source DIRECTLY to a self-contained WebAssembly module — no rustc, no cargo, no wasm-bindgen, no toolchain. Uses the SAME front-end as vm_outcome (crate::ui_bridge::with_parsed_program → crate::vm::Compiler::compile_with_types), so the emitted module runs the IDENTICAL bytecode the VM does; the AOT WebAssembly backend then lowers the whole program to .wasm.
compile_to_wasm_linked: Compile LOGOS source to a LINKED WebAssembly module that uses the real logicaffeine_base::BigInt runtime for arbitrary-precision integer arithmetic: an overflowing integer expression (Show 99999999999 * 99999999999, Show 2 to the power of 200) computes the EXACT big number — matching the VM’s promote-on-overflow — instead of trapping, and Show prints its decimal.
copy_runtime_crates: Copy the runtime crates to the output directory. Copies logicaffeine_data and logicaffeine_system.
core_interp_source: The self-interpreter (coreEval / coreExecBlock) as a first-class committed artifact — the Logos program the genuine P2/P3 projections specialize (Futamura: pe(pe, coreEval) is a compiler). Promoted out of the phase_futamura test fixture so it is a real source file, not a string constant; the fixture now include_str!s this exact file (parity is structural).
core_types_for_pe_source: The canonical inductive type catalog the Futamura projections are built on (CStmt/CExpr/CVal/…). Every statement the tree-walker executes has a C… constructor here; the partial-evaluator dialects dispatch on it. Exposed so cross-tier lock tests can prove this catalog and every tier stay in sync — a statement added to the language must appear here or it is dropped from the projections (see tier_parity_lock.rs).
count_dispatch: The Jones-optimality oracle: count the units of interpreter dispatch remaining in a residual program.
decompile_source_text
display_float_like_logos: Format an f64 exactly as a Show of a Float would — i.e. byte-identical to the tree-walker / VM (RuntimeValue::to_display_string). The direct-WASM host’s print_f64 sink uses this so a Show of a float matches every other engine. Delegating to the real formatter (rather than re-deriving it) makes drift impossible.
encode_program_source: Encode a LogicAffeine program (given as source) into CProgram construction source.
encode_program_source_compact: Compact encoding: inlines simple expressions (literals, variables) to reduce encoding size by ~3x. Same semantics as encode_program_source but produces fewer Let statements.
finish_projection1_residual: The post-processing applied to a raw PE residual (the Any-type safety net + ## Main normalization).
first_parallel_block_independent: Phase 0 test seam: is the first Simultaneously: / Attempt all of the following: block in source made of data-independent branches? None if the program contains no such block.
genuine_projection2_residual: Run genuine PE(pe_source, pe_mini(targetStmts)) and return the specialized compiler residual as LOGOS source code.
genuine_projection3_residual: Run genuine PE(pe_source, pe_bti(targetExpr)) and return the specialized cogen residual as LOGOS source code.
interpret_program: Interpret LOGOS source and return output as a string.
native_export_function_names: The names of functions annotated for the AOT-native tier — ## To <fn> … is exported for native: (HOTSWAP §Axis-3 selectivity). These are the functions largo build --native-functions pre-bundles; everything else stays on VM+JIT.
next_inspect_otherwise_idx: The next Inspect-with-Otherwise flag index (post-increment). Called at the START of each such Inspect’s desugaring in BOTH encoders, so a nested Inspect gets a strictly higher index than its enclosing one — no two live flags ever collide.
optimization_dependencies: The per-program DEPENDENCY edges for source, discovered by evaluating with all optimizations on and probing: disabling a fired optimization dep that makes another fired optimization dependent STOP firing means dependent depended on dep for this program. Stops already explained by a declared requires-cascade (normalize would disable the dependent anyway) are excluded — those are captured by the static registry graph. What remains are the EMERGENT, program-specific dependencies (e.g. dead-code elimination only had work because scalarization produced the dead code): returned as (dependent, dep) keyword pairs, sorted and deduped (deterministic).
optimization_graph: The complete per-program optimization graph from one all-on evaluation: the optimizations that fired, the blockers ((winner, loser) preemptions that occurred), and the emergent dependencies ((dependent, dep) — see optimization_dependencies). Everything the menu-tree needs, baked once per benchmark so the UI never probes in the hot path. All on the AOT codegen path (the one that produces the shown Rust), so the three views agree.
optimization_preemptions: The (winner, loser) precedence decisions the compiler made for source — the conflicts traced at their source (mark_preempted), under the effective config.
optimizations_fired: The decorator keywords of the optimizations that fired for source (registry order), under the default/effective config. One compile, exact.
optimizations_fired_run: The optimizations that fired on the RUN path — the VM/interpreter optimizer (optimize_for_run) — for source. The run-path analogue of optimizations_fired (which traces the AOT codegen path); it surfaces the run-path-only passes — inlining, loop-carried CSE, float strength reduction — the AOT trace cannot see. Respects file-level ## No <opt> decorators.
optimizations_fired_vm: The optimizations that fired while compiling source to VM bytecode — the third path, after AOT codegen and the run-path optimizer. Surfaces the VM-compile-time opts (constant-divisor magic division, VM i32 narrowing) the other two traces cannot see. Compiles only (no execution), so it is fast and needs no program arguments.
optimizations_fired_with: The optimizations that fired for source under an explicit config — re-traces an arbitrary toggle state (turning some off can make others fire). Realised by disabling that config’s off-keywords on the source, then tracing.
optimizations_used: Which optimizations actually FIRE for source: an optimization is “used” when disabling it (via a file-level ## No <X> decorator) changes the generated Rust. Lets the benchmarks UI show, per program, which optimizations it USES vs the full set it CAN use. O(number of optimizations) compiles — intended for per-benchmark, not hot-path, use. Returns the decorator keywords of the firing optimizations (in registry order).
pe_bti_source_text
pe_cdylib_source: The PE as an IN-PROCESS cdylib entry: the identical engine to pe_native_source, but the driver is a single-shot ## To peSpecializeOnce (__prog: CProgram) -> Text that RETURNS the residual instead of writing it to stdout — no readWireProgram/writeWireResidual, no stdin loop. The exported cdylib (build_native_pe_cdylib) decodes the wire bytes to a CProgram in its extern "C" shim and calls this function directly, so the resident server’s process/pipe round-trip is gone. ## Main is a trivial stub (a cdylib never runs it).
pe_combined_source: Assemble the full PE-run source (core types + PE + decompiler + encoded input + driver) — the exact program projection1_source_real_fast executes. Public so differential harnesses can run it on a specific engine.
pe_mini_source_text
pe_native_source
pe_source_text: Returns the source text of the partial evaluator written in LogicAffeine.
program_covered_by_native_builder: Whether the native Core-IR value builder covers program (the fast marshal path is taken, not the interpreter fallback). For test coverage assertions.
program_to_core_wire_bytes: Serialize a program’s Core-IR CProgram (a CProg with its functions and main block) to the plain wire form — the bytes a compile-once native partial evaluator reads on stdin. FAST PATH: [program_to_core] on [WireSink] emits the wire bytes in ONE pass straight from the AST — no intermediate RuntimeValue tree and no second encode_value_raw walk (byte-for-byte identical to the old two-pass form; locked by native_builder_is_byte_identical_to_the_interpreter). FALLBACK: for a construct the native builder does not cover, run the constructor source on the tree-walker (wireBytes). Both are decoded identically by the native binary’s generated CProgram::wire_decode.
program_to_core_wire_bytes_two_pass: The TWO-PASS marshal that program_to_core_wire_bytes replaced: build the full RuntimeValue Core-IR tree ([TreeSink]) then encode_value_raw it. Kept as the correctness ORACLE and the speed BASELINE for the single-pass ([WireSink]) form — the two are byte-identical by construction (WireSink emits exactly what encode_value_raw would), and the single pass avoids the tree allocation + the second encode walk. None if a construct is uncovered.
program_to_core_wire_bytes_via_interpreter: The REFERENCE marshal: always the interpreter path (construct the CProgram via the tree-walker, wireBytes-serialize it). The native builder’s output must be BYTE-IDENTICAL to this — locked by native_pe_wire::native_builder_is_byte_identical_to_the_interpreter. Kept public so the lock can force the reference regardless of native-builder coverage.
projection1_source: First Futamura Projection: PE(interpreter, program) = compiled_program
projection1_source_real: Real Futamura Projection 1: pe(program) = compiled_program
projection1_source_real_fast: In-process variant of projection1_source_real.
projection1_source_real_fast_on_vm: VM-driven twin of projection1_source_real_fast.
projection2_source: Second Futamura Projection: PE(PE, interpreter) = compiler
projection2_source_real: Genuine Futamura Projection 2 via self-application: PE(pe_source, pe_mini(targetStmts))
projection3_source: Third Futamura Projection: PE(PE, PE) = compiler_generator
projection3_source_real: Genuine Futamura Projection 3 via self-application: PE(pe_source, pe_bti(targetStmts))
quote_pe_source: Encodes the partial evaluator as CProgram construction source code.
run_genuine_p2_on_target: Run genuine P2 on a specific target: PE(pe_source, pe_mini(target))
run_genuine_p3_on_target: Run genuine P3 on a specific target: PE(pe_source, pe_bti(target))
run_logos_source: Compile and run LOGOS source, returning stdout.
run_native_pe: Specialize program on the compile-once native PE — the DEFAULT entry and the pit of success. Prefers the IN-PROCESS cdylib path (run_native_pe_inprocess; no process/pipe boundary) and falls back to the resident server (run_native_pe_server) only if the in-process build/load fails (e.g. no toolchain), so the fast+correct route is what every caller gets by default. The two paths are proven byte-identical by native_pe_wire.
run_native_pe_inprocess: Specialize program on the compile-once native PE, IN PROCESS: marshal it to wire bytes, hand them to the dlopened PE cdylib’s logos_pe_specialize as a plain function call, and reclaim the residual bytes. No child process, no pipe — the ~59 µs IPC round-trip of the resident server is gone; all that remains is the host marshal + native peBlock + one FFI call. The cdylib is built + loaded ONCE per thread and cached. Residual-identity with the server / tree-walker is locked by native_pe_wire.
run_native_pe_server
rustc_check: Run rustc’s analysis over a LOGOS program and translate every finding back to English with real user-source spans — the flycheck engine.
rustc_check_artifacts: Compile for CHECKING: the same front as compile_program_full but with the optimizer off and mapped codegen, so every generated line ties back to its statement span and the map stays 1:1 (the optimizer may fuse or drop statements; a check build has no use for its output anyway).
send_check_source: Phase 4 (work/FINISH_INTERPRETER.md): run the Send/escape analysis over a program.
stmts_to_core_wire_bytes: The PARSE-FREE marshal: emit a program’s Core-IR wire bytes directly from an ALREADY-PARSED AST, skipping the lex + discovery + parse_program that dominates program_to_core_wire_bytes (~90% of a specialization). An integrated caller that parsed the program once for compilation reuses that AST here to feed the native PE at a fraction of the cost. variant_fields maps each variant constructor name to its ordered field names (the caller derives it from its type registry, as [program_to_core] does). None if a construct is uncovered. Byte-identical to program_to_core_wire_bytes on the same program — same [build_core]/[WireSink] spine.
tw_outcome: Run source on the TREE-WALKER (the oracle), capturing partial output and the error, if any. This entry point always uses the tree-walker — the dispatching interpret_for_ui_sync now runs the VM for the sync path, and the differential suites need the oracle by itself.
tw_outcome_with_args: tw_outcome with the program argument vector for the args() system native (full argv; index 0 is the program name) — the seam the benchmark corpus differential uses to run main.lg programs that read their size from argv.
verify_no_overhead_source: Verify that a LogicAffeine program has no interpretive overhead.
vm_outcome: Run source on the bytecode VM, capturing partial output and the error, if any. Same front-end as tw_outcome.
vm_outcome_bg: vm_outcome_with_args driven through the BACKGROUND native compiler (HOTSWAP §6): hot functions compile on a worker thread off the interpreter’s path. Requires a process-installed tier (the P8 gate installs one first). The compiled chain is identical to the synchronous one — only WHEN it is produced differs — so output must match the synchronous/tree-walker engines exactly.
vm_outcome_concurrent: The VM outcome for a CONCURRENT program — drives the bytecode VM on the cooperative scheduler (VmTask on run_with_seed, seed 0) exactly as the browser/playground does, so a Pipe/task/ Select program runs on the VM byte-identical to the tree-walker’s async path. The plain vm_outcome errors on any concurrency op (requires the scheduler driver); this is the oracle the AOT concurrency lock compares against so tw == VM == AOT is verifiable for the concurrent set.
vm_outcome_net: The VM outcome for a peer-NETWORKING program — drives the resumable bytecode VM and services each VmBlock::Net* through the shared LOCAL NetInbox (the same inbox the tree-walker’s offline mode uses), so Listen/Send/Sync/PeerAgent run on the VM byte-identical to the tree-walker. The AOT networking lock compares against this so tw == VM == AOT holds for the local-mode net set.
vm_outcome_tiered: vm_outcome_with_args, but the statements pass through the run-path optimizer at an explicit hotness tier (HOTSWAP §4) before reaching the VM — the seam the tier_invariance gate drives at every tier T0..T3 to assert byte-identical output. tier = Tier::T3 reproduces today’s optimized run path.
vm_outcome_with_args: vm_outcome with the program argument vector and an optional private native tier (so a differential test can observe THAT program’s JIT compile counters in isolation — the process-wide tier’s counters are shared by every test in the binary).
vm_run_source: Parse LOGOS source and execute it on the register bytecode VM, returning the captured output. Uses the SAME front-end as the tree-walker (crate::ui_bridge::with_parsed_program: lex → MWE → discovery → parse), so differential tests compare two engines on one program, never two programs. Returns Err for programs outside the VM’s supported subset (the compiler reports vm: unsupported …).
write_cargo_project: Write a generated program as a runnable cargo project: src/main.rs, Cargo.toml (runtime path-deps + user ## Requires), .cargo/config.toml, and the staged runtime crates. Shared by compile_to_dir and rustc_check.

Module compile

Module compile Copy item path

§Pipeline Overview

§Compilation Functions

§Examples

§Basic Compilation

§With Ownership Checking

Re-exports§

Structs§

Enums§

Functions§

Module compile