7 releases (breaking)
0.8.0 | May 2, 2023 |
---|---|
0.7.0 | Apr 16, 2023 |
0.6.0 | Feb 13, 2023 |
0.5.0 | Dec 4, 2022 |
0.1.0 | Feb 7, 2021 |
#307 in Parser implementations
1,548 downloads per month
445KB
11K
SLoC
LEMON parser generator modified to generate Rust code.
Lemon source and SQLite3 grammar were last synced as of May 2022.
Unsupported
Unsupported Grammar syntax
%token_destructor
: Code to execute to destroy token data%default_destructor
: Code for the default non-terminal destructor%destructor
: Code which executes whenever this symbol is popped from the stack during error processing
https://www.codeproject.com/Articles/1056460/Generating-a-High-Speed-Parser-Part-Lemon https://www.sqlite.org/lemon.html
SQLite
SQLite lexer and SQLite parser have been ported from C to Rust. The parser generates an AST.
Lexer/Parser:
- Keep track of position (line, column).
- Streamable (stop at the end of statement).
- Resumable (restart after the end of statement).
Lexer and parser have been tested with the following scripts:
- https://github.com/bkiers/sqlite-parser/tree/master/src/test/resources
- https://github.com/codeschool/sqlite-parser/tree/master/test/sql/official-suite which can be updated with script in https://github.com/codeschool/sqlite-parser/tree/master/test/misc
TODO:
- Check generated AST (reparse/reinject)
- If a keyword in double quotes is used in a context where it cannot be resolved to an identifier but where a string literal is allowed, then the token is understood to be a string literal instead of an identifier.
- Tests
- Do not panic while parsing
- CREATE VIRTUAL TABLE args
- Zero copy (at least tokens)
Unsupported by Rust
#line
directive
API change
- No
ParseAlloc
/ParseFree
anymore
Features not tested
- NDEBUG
- YYNOERRORRECOVERY
- YYERRORSYMBOL
To be fixed
- RHS are moved. Maybe it is not a problem if they are always used once. Just add a check in lemon...
%extra_argument
is not supported.- Terminal symbols generated by lemon should be dumped in a specified file.
Raison d'être
-
lemon_rust does the same thing but with an old version of
lemon
. And it seems not possible to useyystack
as a stack because items may be access randomly and thetop+1
item can be used. -
lalrpop would be the perfect alternative but it does not support fallback/streaming (see this issue) and compilation/generation is slow.
Dependencies
~1–1.5MB
~28K SLoC