Lib.rs

Given source strings, MiTeX Lexer provides a sequence of tokens

The core of the lexer is [Lexer<'a, S>] which receives a string &'a str and a TokenStream trait object S, then it provides public methods to peek and bump the token stream.

It has two main lexer implementations:

• [Lexer<()>]: provides plain tokens
  • See TokenStream for implementation
• Lexer<MacroEngine>: provides tokens with macro expansion
  • See MacroEngine for implementation