4 releases (breaking)
0.4.0 | May 16, 2024 |
---|---|
0.3.0 | May 14, 2024 |
0.2.0 | May 14, 2024 |
0.1.0 | May 12, 2024 |
#291 in Programming languages
19KB
460 lines
Language Pack - Scanner
Language Pack - Scanner, is scanning/lexing Rust library, useful for interpreter/compiler applications. Library is intended to be highly customizable and easy to learn, it recognizes every character aviable on the keyboard and keywords you can configure, and returns an array of tokens, you can use for building AST.
installation
Run the following Cargo command in your project directory:
cargo add lp-pack-scanner
Or add the following line to your Cargo.toml:
lp-pack-scanner = "0.4.0"
Example
use std::collections::HashMap;
use lp_pack_scanner::{Config, Scanner, Token};
fn main() {
let tokens = Scanner::scan(Config {
input: "let x = 5".to_string(),
keywords: HashMap::from([("let", lp_pack_scanner::Token::Variable)]),
allow_unknown: Some(true),
errors: None,
comments: None,
benchmark: false,
});
println!("{:?}", tokens);
}
/*
[OutputToken { token: Variable, literal: None, lexeme: "let", line: 1 }, OutputToken { token: Identifier, literal: None, lexeme: "x", line: 1 }, OutputToken { token: Equal, literal: None, lexeme: "=", line: 1 }, OutputToken { token: Number, literal: Number(5.0), lexeme: "5", line: 1 }, OutputToken { token: EndOfFile, literal: None, lexeme: "", line: 1 }]
*/
Documentation
soon