48 releases

0.14.2 Sep 18, 2024
0.14.0 Feb 7, 2024
0.13.0 Apr 10, 2023
0.12.1 Jun 8, 2022
0.7.3 Nov 30, 2018

#3 in Parser tooling

Download history 92311/week @ 2024-08-04 99700/week @ 2024-08-11 93418/week @ 2024-08-18 94285/week @ 2024-08-25 84358/week @ 2024-09-01 102342/week @ 2024-09-08 96730/week @ 2024-09-15 109609/week @ 2024-09-22 125114/week @ 2024-09-29 120537/week @ 2024-10-06 112322/week @ 2024-10-13 143976/week @ 2024-10-20 125406/week @ 2024-10-27 117628/week @ 2024-11-03 116586/week @ 2024-11-10 123934/week @ 2024-11-17

490,838 downloads per month
Used in 759 crates (263 directly)

MIT/Apache

1.5MB
693 lines

Logos logo

Logos

Test Crates.io version shield Docs Crates.io license shield

Create ridiculously fast Lexers.

Logos has two goals:

  • To make it easy to create a Lexer, so you can focus on more complex problems.
  • To make the generated Lexer faster than anything you'd write by hand.

To achieve those, Logos:

Example

 use logos::Logos;

 #[derive(Logos, Debug, PartialEq)]
 #[logos(skip r"[ \t\n\f]+")] // Ignore this regex pattern between tokens
 enum Token {
     // Tokens can be literal strings, of any length.
     #[token("fast")]
     Fast,

     #[token(".")]
     Period,

     // Or regular expressions.
     #[regex("[a-zA-Z]+")]
     Text,
 }

 fn main() {
     let mut lex = Token::lexer("Create ridiculously fast Lexers.");

     assert_eq!(lex.next(), Some(Ok(Token::Text)));
     assert_eq!(lex.span(), 0..6);
     assert_eq!(lex.slice(), "Create");

     assert_eq!(lex.next(), Some(Ok(Token::Text)));
     assert_eq!(lex.span(), 7..19);
     assert_eq!(lex.slice(), "ridiculously");

     assert_eq!(lex.next(), Some(Ok(Token::Fast)));
     assert_eq!(lex.span(), 20..24);
     assert_eq!(lex.slice(), "fast");

     assert_eq!(lex.next(), Some(Ok(Token::Text)));
     assert_eq!(lex.slice(), "Lexers");
     assert_eq!(lex.span(), 25..31);

     assert_eq!(lex.next(), Some(Ok(Token::Period)));
     assert_eq!(lex.span(), 31..32);
     assert_eq!(lex.slice(), ".");

     assert_eq!(lex.next(), None);
 }

For more examples and documentation, please refer to the Logos handbook or the crate documentation.

How fast?

Ridiculously fast!

test identifiers                       ... bench:         647 ns/iter (+/- 27) = 1204 MB/s
test keywords_operators_and_punctators ... bench:       2,054 ns/iter (+/- 78) = 1037 MB/s
test strings                           ... bench:         553 ns/iter (+/- 34) = 1575 MB/s

Acknowledgements

Thank you

Logos is very much a labor of love. If you find it useful, consider getting me some coffee. ☕

If you'd like to contribute to Logos, then consider reading the Contributing guide.

Contributing

Logos welcome any kind of contribution: bug reports, suggestions, or new features!

Please use the issues or pull requests tabs, when appropriate.

Releasing a new version

[!NOTE]

This section is only useful to Logos' maintainers.

First, make sure you are logged-in https://crates.io with: cargo login. If you don't have write access to Logos' crates, you can still perform steps 1-4, and ask a maintainer with accesses to perform step 5.

This project uses cargo-release to publish all packages with more ease. Note that, by default, every command runs in dry mode, and you need to append --execute to actually perform the action.

Here are the following steps to release a new version:

  1. create a branch release-x.y.z from the master branch;
  2. run and commit cargo release version --workspace <LEVEL>;
  3. run and commit cargo release replace --workspace;
  4. push your branch and create a pull request;
  5. and, once your branch was merged to master, run the following:
    cargo release publish --package logos-codegen
    cargo release publish --package logos-derive
    cargo release publish --package logos-cli
    cargo release publish --package logos
    

And voilà!

License

This code is distributed under the terms of both the MIT license and the Apache License (Version 2.0), choose whatever works for you.

See LICENSE-APACHE and LICENSE-MIT for details.

Dependencies