Skip to main content

tp_decode_layer

Function tp_decode_layer 

Source
pub fn tp_decode_layer(
    spec: &ModelSpec,
    l: usize,
    dir: &str,
    ctx: usize,
    pos: usize,
    tp: usize,
) -> Result<TpAuthored, NotYet>
Expand description

Layer l of a tensor-parallel decode step alone: its input and position declared, the rotary tables for an attention layer, then the split layer.