pub fn encode_tokens(
tokens: &[Token],
line_index: &LineIndex,
) -> Vec<SemanticToken>Expand description
Convert a token stream to LSP semantic tokens (delta-encoded) using the
base part-of-speech layer only. encode_document_tokens is the
resolution-aware entry point.