47 releases

0.15.0 Dec 7, 2024
0.14.2 Sep 18, 2024
0.14.0 Feb 7, 2024
0.13.0 Apr 10, 2023
0.7.3 Nov 30, 2018

#25 in #lexical

Download history 120958/week @ 2024-10-04 114379/week @ 2024-10-11 135883/week @ 2024-10-18 133059/week @ 2024-10-25 114496/week @ 2024-11-01 116805/week @ 2024-11-08 121041/week @ 2024-11-15 114356/week @ 2024-11-22 128649/week @ 2024-11-29 132977/week @ 2024-12-06 126586/week @ 2024-12-13 65057/week @ 2024-12-20 61244/week @ 2024-12-27 109979/week @ 2025-01-03 130703/week @ 2025-01-10 109275/week @ 2025-01-17

418,003 downloads per month
Used in 47 crates (6 directly)

MIT/Apache

170KB
4K SLoC

Logos logo

Logos

Test Crates.io version shield Docs Crates.io license shield

Create ridiculously fast Lexers.

Logos has two goals:

  • To make it easy to create a Lexer, so you can focus on more complex problems.
  • To make the generated Lexer faster than anything you'd write by hand.

To achieve those, Logos:

Example

use logos::Logos;

#[derive(Logos, Debug, PartialEq)]
#[logos(skip r"[ \t\n\f]+")] // Ignore this regex pattern between tokens
enum Token {
    // Tokens can be literal strings, of any length.
    #[token("fast")]
    Fast,

    #[token(".")]
    Period,

    // Or regular expressions.
    #[regex("[a-zA-Z]+")]
    Text,
}

fn main() {
    let mut lex = Token::lexer("Create ridiculously fast Lexers.");

    assert_eq!(lex.next(), Some(Ok(Token::Text)));
    assert_eq!(lex.span(), 0..6);
    assert_eq!(lex.slice(), "Create");

    assert_eq!(lex.next(), Some(Ok(Token::Text)));
    assert_eq!(lex.span(), 7..19);
    assert_eq!(lex.slice(), "ridiculously");

    assert_eq!(lex.next(), Some(Ok(Token::Fast)));
    assert_eq!(lex.span(), 20..24);
    assert_eq!(lex.slice(), "fast");

    assert_eq!(lex.next(), Some(Ok(Token::Text)));
    assert_eq!(lex.slice(), "Lexers");
    assert_eq!(lex.span(), 25..31);

    assert_eq!(lex.next(), Some(Ok(Token::Period)));
    assert_eq!(lex.span(), 31..32);
    assert_eq!(lex.slice(), ".");

    assert_eq!(lex.next(), None);
}

For more examples and documentation, please refer to the Logos handbook or the crate documentation.

How fast?

Ridiculously fast!

test identifiers                       ... bench:         647 ns/iter (+/- 27) = 1204 MB/s
test keywords_operators_and_punctators ... bench:       2,054 ns/iter (+/- 78) = 1037 MB/s
test strings                           ... bench:         553 ns/iter (+/- 34) = 1575 MB/s

Acknowledgements

Thank you

Logos is very much a labor of love. If you find it useful, consider getting me some coffee. ☕

If you'd like to contribute to Logos, then consider reading the Contributing guide.

Contributing

Logos welcome any kind of contribution: bug reports, suggestions, or new features!

Please use the issues or pull requests tabs, when appropriate.

To release a new version, follow the RELEASE-PROCESS

License

This code is distributed under the terms of both the MIT license and the Apache License (Version 2.0), choose whatever works for you.

See LICENSE-APACHE and LICENSE-MIT for details.

Dependencies

~1–1.9MB
~51K SLoC