Skip to main content

decode_layer

Function decode_layer 

Source
pub fn decode_layer(
    spec: &ModelSpec,
    l: usize,
    dir: &str,
    ctx: usize,
    pos: usize,
) -> Authored
Expand description

Layer l of a decode step alone, its attention reading a KV cache of ctx positions up to pos.