Skip to main content

compile_to_cuda_forward_weighted

Function compile_to_cuda_forward_weighted 

Source
pub fn compile_to_cuda_forward_weighted(
    source: &str,
    weights: &HashMap<String, (String, usize)>,
) -> Result<String, String>
Expand description

compile_to_cuda_forward with a WEIGHT MANIFEST: var name → (file path, float count). Any tensor variable named in the manifest is loaded from that file by the emitted host at runtime instead of being inlined — the mechanism the real-8B forward needs, since the Qwen weights are gigabytes (extracted from the GGUF checkpoint). Every other input still uses its literal Set values / the deterministic fill.