#nom #parsing #generator #pest #macro

macro nom-rule

A procedural macro for defining nom combinators in simple DSL

7 releases

0.3.0 May 8, 2022
0.2.2 Apr 27, 2022
0.2.1 Feb 6, 2022
0.1.4 Feb 3, 2022
0.1.1 Jan 31, 2022

#514 in Procedural macros

Download history 7335/week @ 2023-02-10 6698/week @ 2023-02-17 2678/week @ 2023-02-24 3815/week @ 2023-03-03 3919/week @ 2023-03-10 3286/week @ 2023-03-17 3183/week @ 2023-03-24 2726/week @ 2023-03-31 2933/week @ 2023-04-07 1604/week @ 2023-04-14 2157/week @ 2023-04-21 268/week @ 2023-04-28 482/week @ 2023-05-05 752/week @ 2023-05-12 850/week @ 2023-05-19 726/week @ 2023-05-26

2,876 downloads per month

MIT license

26KB
545 lines

nom-rule

Documentation Crates.io LICENSE

A procedural macro for defining nom combinators in simple DSL. Requires nom v5.0+.

Dependencies

[dependencies]
nom = "7"
nom-rule = "0.2"

Syntax

The procedural macro rule! provided by this crate is designed for the ease of writing grammar spec as well as to improve maintainability, it follows these simple rules:

  1. TOKEN: match the token by token kind. You should provide a parser to eat the next token if the token kind matched. it will get expanded into match_token(TOKEN).
  2. ";": match the token by token text. You should provide a parser to eat the next token if the token text matched. it will get expanded into match_text(";") in this example.
  3. #fn_name: an external nom parser function. In the example above, ident is a predefined parser for identifiers.
  4. a ~ b ~ c: a sequence of parsers to take one by one. It'll get expanded into nom::sequence::tuple.
  5. (...)+: one or more repeated patterns. It'll get expanded into nom::multi::many1.
  6. (...)*: zero or more repeated patterns. It'll get expanded into nom::multi::many0.
  7. (...)?: Optional parser. It'll get expanded into nom::combinator::opt.
  8. a | b | c: Choices between a, b, and c. It'll get expanded into nom::branch::alt.
  9. &a: Peek. It'll get expanded into nom::combinator::peek(a). Note that it doesn't consume the input.
  10. !a: Negative predicate. It'll get expanded into nom::combinator::not. Note that it doesn't consume the input.
  11. ^a: Cut parser. It'll get expanded into nom::combinator::cut.
  12. ... : "description": Context description for error reporting. It'll get expanded into nom::error::context.

Example

Define match_text parser and match_token parser for your custom token type. You can use nom::combinator::fail as match_token if your parser use &str or &[u8] as input because you won't match on token kinds.

#[derive(Clone, Debug, PartialEq)]
struct Token<'a> {
    kind: TokenKind,
    text: &'a str,
    span: Span,
}

#[derive(Clone, Copy, Debug, PartialEq)]
enum TokenKind {
    Whitespace,

    // Keywords
    CREATE,
    TABLE,

    // Symbols
    LParen,
    RParen,
    Semicolon,
    Comma,

    Ident,
}

fn match_text<'a, Error: ParseError<Input<'a>>>(
    text: &'a str,
) -> impl FnMut(Input<'a>) -> IResult<Input<'a>, &'a Token<'a>, Error> {
    move |i| satisfy(|token: &Token<'a>| token.text == text)(i)
}

fn match_token<'a, Error: ParseError<Input<'a>>>(
    kind: TokenKind,
) -> impl FnMut(Input<'a>) -> IResult<Input<'a>, &'a Token<'a>, Error> {
    move |i| satisfy(|token: &Token<'a>| token.kind == kind)(i)
}

Then give the two parser to nom_rule::rule! by wrapping it into a custom macro:

macro_rules! rule {
    ($($tt:tt)*) => { 
        nom_rule::rule!($crate::match_text, $crate::match_token, $($tt)*)
    }
}

To define a parser for the SQL of creating table:

let mut rule = rule!(
    CREATE ~ TABLE ~ #ident ~ ^"(" ~ (#ident ~ #ident ~ ","?)* ~ ")" ~ ";" : "CREATE TABLE statement"
);

It will get expanded into:

let mut rule = 
    nom::error::context(
        "CREATE TABLE statement",
        nom::sequence::tuple((
            (crate::match_token)(CREATE),
            (crate::match_token)(TABLE),
            ident,
            (nom::combinator::cut(crate::match_text)("(")),
            nom::multi::many0(nom::sequence::tuple((
                ident,
                ident,
                nom::combinator::opt((crate::match_text)(",")),
            ))),
            (crate::match_text)(")"),
            (crate::match_text)(";"),
        ))
    );

Auto Sequence (nightly only)

nom-rule is able to automatically insert ~ in the rule when necessary so that you get the example above working the same as the following:

let mut rule = rule!(
    CREATE TABLE #ident "(" (#ident #ident ","?)* ")" ";" : "CREATE TABLE statement"
);

To enable this feature, you need to use a nightly channel rust complier, and add this to the Cargo.toml:

nom-rule = { version = "0.2", features = ["auto-sequence"] }

Dependencies

~2MB
~47K SLoC