#lua #glr #parser

lua_tokenizer

tokenizer for lua language

1 unstable release

0.1.0 Sep 8, 2024

#1448 in Parser implementations

Download history 188/week @ 2024-09-08

188 downloads per month
Used in lua_parser

MIT/Apache

76KB
2K SLoC

lua_rust

lua syntax parser in Rust

project structure

  • tokenizer: tokenizing lua code string
  • parser: parsing tokenized lua code into AST
  • exec: executable version of the parser

Cargo Features

  • 32bit: use 32bit integer and float for lua numeric type
  • diag: enable to_diag() function for ParseError

how to run

$ cargo run <source_file.lua>

will print the pretty-formatted Debug output of the AST ( "{:#?}" )


lib.rs:

let source = " <source code here> ";

let tokenizer = Tokenizer::new(source);
// tokenizer itself is a lazy iterator.
for token in tokenizer {
    match token {
        Ok(token) => {
            // do something with token
        }
        Err(e) => {
            print!("Tokenize Error: {}", e);
        }
    }
}

Dependencies

~0–6.5MB
~37K SLoC