Skip to main content

len_utf8

logicaffeine_compile::model::tokenizer::unicode

Function len_utf8 

Source 
pub fn len_utf8(lead: u8) -> usize
Expand description

unicode_len_utf8: the sequence length a lead byte announces (a continuation byte counts as one).