Skip to main content

encode_document_tokens

Function encode_document_tokens 

Source
pub fn encode_document_tokens(doc: &DocumentState) -> Vec<SemanticToken>
Expand description

Encode a document’s tokens with the full resolution overlay.

The base layer is part-of-speech classification (classify_token); the overlay upgrades identifiers through the SymbolIndex — a name that resolves to a parameter IS a parameter, at declaration and at every reference — and adds the modifier facts: DECLARATION only where token.span == definition.span, READONLY for immutable Lets, MODIFICATION on write sites, DEFAULT_LIBRARY on stdlib prelude names. Prose inside ## Note/## Example blocks recedes to COMMENT.