8 releases

0.1.8 Jun 1, 2020
0.1.7 May 15, 2019
0.1.6 Apr 20, 2019
0.1.5 Mar 9, 2019
0.1.3 Jan 25, 2019

#2310 in Parser implementations

MIT license

80KB
2K SLoC

Javascript Lexer

Build Status Documentation Creates License

Javscript lexer implements high performance lexer of javscript as defined by ECMAScript 9

Output of the lexer together with description can be found in docs

Installation

add to cargo.toml

[dependencies]
javascript_lexer = "0.1"

Example

Lexer is invoked by running

extern crate javascript_lexer;

use javascript_lexer::Lexer;

fn main() {
    Lexer::parse_tokens("javascript");
}

lib.rs:

Lexer module consumes text input and produces linear stream of tokens.

Tokens are defined in token

Dependencies

~3MB
~59K SLoC