5 releases
0.2.0 | Sep 16, 2021 |
---|---|
0.1.3 | Oct 21, 2020 |
0.1.2 | Oct 6, 2020 |
0.1.1 | Oct 3, 2020 |
0.1.0 | Oct 2, 2020 |
#417 in Web programming
346 downloads per month
Used in 5 crates
(3 directly)
150KB
3.5K
SLoC
An extremely fast, lookup table based, ECMAScript lexer which yields SyntaxKind tokens used by the rslint_parse parser.
For the purposes of error recovery, tokens may have an error attached to them, which is reflected in the Iterator Item.
The lexer will also yield COMMENT
and WHITESPACE
tokens.
The lexer operates on raw bytes to take full advantage of lookup table optimizations, these bytes must be valid utf8,
therefore making a lexer from a &[u8]
is unsafe since you must make sure the bytes are valid utf8.
Do not use this to learn how to lex JavaScript, this is just needlessly fast and demonic because i can't control myself :)
basic ANSI syntax highlighting is also offered through the highlight
feature.
Warning ⚠️
>>
and >>>
are not emitted as single tokens, they are emitted as multiple >
tokens. This is because of
TypeScript parsing and productions such as T<U<N>>
Dependencies
~1.5MB
~30K SLoC