1 unstable release
0.1.0 | Feb 21, 2022 |
---|
#292 in Parser tooling
17KB
264 lines
logos-nom-bridge
A logos Lexer
wrapper than can be used
as an input for nom. This makes it very simple
to parse tokens lexed with logos.
Usage
See the documentation.
Contributing
You can contribute by filing issues or sending pull requests. If you have questions, please create an issue.
License
Dual-licensed under the MIT license or the Apache 2.0 license.
lib.rs
:
logos-nom-bridge
A logos::Lexer
wrapper than can be used as an input for
nom.
Simple example
// First, create a `logos` lexer:
#[derive(Clone, Debug, PartialEq, Eq, logos::Logos)]
enum Token {
#[token("+")]
Plus,
#[token("-")]
Minus,
#[regex(r"-?[0-9]+", |lex| lex.slice().parse())]
Number(i64),
#[error]
#[regex(r"[ \t\n\f]+", logos::skip)]
Error,
}
// Then, write a nom parser that accepts a `Tokens<'_, Token>` as input:
use logos_nom_bridge::Tokens;
type Input<'source> = Tokens<'source, Token>;
#[derive(Debug, PartialEq, Eq)]
enum Op {
Number(i64),
Addition(Box<(Op, Op)>),
Subtraction(Box<(Op, Op)>),
}
fn parse_expression(input: Input<'_>) -> nom::IResult<Input<'_>, Op> {
#
#
// zip
}
// Finally, you can use it to parse a string:
let input = "10 + 3 - 4";
let tokens = Tokens::new(input);
let (rest, parsed) = parse_expression(tokens).unwrap();
assert!(rest.is_empty());
assert_eq!(
parsed,
Op::Addition(Box::new((
Op::Number(10),
Op::Subtraction(Box::new((
Op::Number(3),
Op::Number(4),
))),
))),
)
Macros
You can implement nom::Parser
for your token type with the token_parser
macro:
#
logos_nom_bridge::token_parser!(token: Token);
If some enum variants of your token type contain data, you can implement a nom::Parser
for them using the data_variant_parser
macro:
#
#[derive(Clone, Debug, PartialEq, Eq, logos::Logos)]
enum Token {
#[regex(r"-?[0-9]+", |lex| lex.slice().parse())]
Number(i64),
// etc.
}
logos_nom_bridge::data_variant_parser! {
fn parse_number(input) -> Result<Op>;
pattern = Token::Number(n) => Op::Number(n);
}
Dependencies
~2.5MB
~20K SLoC