#tokenization #parser #proc-macro #macro-derive #utilities #generate

derive-finite-automaton

Procedural macro for generating finite automaton

5 releases

0.2.0 Mar 1, 2024
0.1.3 Sep 20, 2023
0.1.2 Sep 20, 2023
0.1.1 Sep 19, 2023
0.1.0 Dec 6, 2021

#119 in Parser tooling

Download history 17/week @ 2024-03-14 5/week @ 2024-03-21 26/week @ 2024-03-28 40/week @ 2024-04-04 9/week @ 2024-04-11 8/week @ 2024-04-18 5/week @ 2024-04-25 4/week @ 2024-05-02 8/week @ 2024-05-09 15/week @ 2024-05-16 21/week @ 2024-05-23 30/week @ 2024-05-30 19/week @ 2024-06-06 21/week @ 2024-06-13 25/week @ 2024-06-20 6/week @ 2024-06-27

74 downloads per month
Used in 4 crates (via ezno-parser)

MIT license

7KB
81 lines

derive finite automaton

crates.io badge docs.rs badge

A procedural macro for building a finite automaton. Main use is for lexing multiple character wide tokens see derive-finite-automaton/examples/main.rs.

Run example:

cd derive-finite-automaton
cargo run --example main

View example macro expansion (requires cargo-expand):

cd derive-finite-automaton
cargo expand --example main

Example

use derive_finite_automaton::{FiniteAutomata, FiniteAutomataConstructor};

#[derive(FiniteAutomataConstructor, Debug, PartialEq)]
#[automaton_mappings(
    "{" => Tokens::OpenBrace,
    "}" => Tokens::CloseBrace,
    "=" => Tokens::Assign,
    "=>" => Tokens::ArrowFunction,
    "==" => Tokens::Equal,
    "===" => Tokens::StrictEqual,
    "." => Tokens::Dot,
    "..." => Tokens::Spread,
)]
pub enum Tokens {
    OpenBrace,
    CloseBrace,
    ArrowFunction,
    Equal,
    StrictEqual,
    Assign,
    Dot,
    Spread,
}

You can add conditional mappings with the following

use derive_finite_automaton::FiniteAutomataConstructor;

#[derive(Debug, FiniteAutomataConstructor)]
#[automaton_mappings(
    "{" => Tokens::OpenBrace,
    "}" => Tokens::CloseBrace,
    "=>" => Tokens::ArrowFunction,
    "==" => Tokens::Equal,
    "===" => Tokens::StrictEqual,
    "=" => Tokens::Assign,
    // Some mapping
    "." => Tokens::Dot,
)]
#[cfg_attr(feature = "special", automaton_mappings(
    ".?." => Tokens::Magic,
))]
pub enum Tokens {
    OpenBrace,
    CloseBrace,
    ArrowFunction,
    Equal,
    StrictEqual,
    Assign,
    Dot,
    #[cfg(feature = "special")]
    Magic,
}

Dependencies

~0.3–0.8MB
~19K SLoC