42 releases

0.14.0 Feb 7, 2024
0.13.0 Apr 10, 2023
0.12.1 Jun 8, 2022
0.12.0 Feb 1, 2021
0.7.3 Nov 30, 2018

#321 in Parser tooling

Download history 60046/week @ 2023-11-28 66767/week @ 2023-12-05 59619/week @ 2023-12-12 42958/week @ 2023-12-19 23982/week @ 2023-12-26 51780/week @ 2024-01-02 61029/week @ 2024-01-09 65526/week @ 2024-01-16 62189/week @ 2024-01-23 58428/week @ 2024-01-30 66633/week @ 2024-02-06 64478/week @ 2024-02-13 63026/week @ 2024-02-20 76197/week @ 2024-02-27 81478/week @ 2024-03-05 22805/week @ 2024-03-12

251,086 downloads per month
Used in 22 crates (5 directly)

MIT/Apache

165KB
4K SLoC

Logos logo

Logos

Test Crates.io version shield Docs Crates.io license shield

Create ridiculously fast Lexers.

Logos has two goals:

  • To make it easy to create a Lexer, so you can focus on more complex problems.
  • To make the generated Lexer faster than anything you'd write by hand.

To achieve those, Logos:

Example

 use logos::Logos;

 #[derive(Logos, Debug, PartialEq)]
 #[logos(skip r"[ \t\n\f]+")] // Ignore this regex pattern between tokens
 enum Token {
     // Tokens can be literal strings, of any length.
     #[token("fast")]
     Fast,

     #[token(".")]
     Period,

     // Or regular expressions.
     #[regex("[a-zA-Z]+")]
     Text,
 }

 fn main() {
     let mut lex = Token::lexer("Create ridiculously fast Lexers.");

     assert_eq!(lex.next(), Some(Ok(Token::Text)));
     assert_eq!(lex.span(), 0..6);
     assert_eq!(lex.slice(), "Create");

     assert_eq!(lex.next(), Some(Ok(Token::Text)));
     assert_eq!(lex.span(), 7..19);
     assert_eq!(lex.slice(), "ridiculously");

     assert_eq!(lex.next(), Some(Ok(Token::Fast)));
     assert_eq!(lex.span(), 20..24);
     assert_eq!(lex.slice(), "fast");

     assert_eq!(lex.next(), Some(Ok(Token::Text)));
     assert_eq!(lex.slice(), "Lexers");
     assert_eq!(lex.span(), 25..31);

     assert_eq!(lex.next(), Some(Ok(Token::Period)));
     assert_eq!(lex.span(), 31..32);
     assert_eq!(lex.slice(), ".");

     assert_eq!(lex.next(), None);
 }

For more examples and documentation, please refer to the Logos handbook or the crate documentation.

How fast?

Ridiculously fast!

test identifiers                       ... bench:         647 ns/iter (+/- 27) = 1204 MB/s
test keywords_operators_and_punctators ... bench:       2,054 ns/iter (+/- 78) = 1037 MB/s
test strings                           ... bench:         553 ns/iter (+/- 34) = 1575 MB/s

Acknowledgements

Thank you

Logos is very much a labor of love. If you find it useful, consider getting me some coffee. ☕

If you'd like to contribute to Logos, then consider reading the Contributing guide.

License

This code is distributed under the terms of both the MIT license and the Apache License (Version 2.0), choose whatever works for you.

See LICENSE-APACHE and LICENSE-MIT for details.

Dependencies

~1.1–2MB
~52K SLoC