logos-nom-bridge

A logos Lexer wrapper than can be used as an input for nom. This makes it very simple to parse tokens lexed with logos.

Usage

See the documentation.

Contributing

You can contribute by filing issues or sending pull requests. If you have questions, please create an issue.

License

Dual-licensed under the MIT license or the Apache 2.0 license.

lib.rs:

logos-nom-bridge

A logos::Lexer wrapper than can be used as an input for nom.

Simple example

// First, create a `logos` lexer:

#[derive(Clone, Debug, PartialEq, Eq, logos::Logos)]
enum Token {
    #[token("+")]
    Plus,

    #[token("-")]
    Minus,

    #[regex(r"-?[0-9]+", |lex| lex.slice().parse())]
    Number(i64),

    #[error]
    #[regex(r"[ \t\n\f]+", logos::skip)]
    Error,
}

// Then, write a nom parser that accepts a `Tokens<'_, Token>` as input:

use logos_nom_bridge::Tokens;

type Input<'source> = Tokens<'source, Token>;

#[derive(Debug, PartialEq, Eq)]
enum Op {
    Number(i64),
    Addition(Box<(Op, Op)>),
    Subtraction(Box<(Op, Op)>),
}

fn parse_expression(input: Input<'_>) -> nom::IResult<Input<'_>, Op> {
#
#
    // zip
}

// Finally, you can use it to parse a string:

let input = "10 + 3 - 4";
let tokens = Tokens::new(input);

let (rest, parsed) = parse_expression(tokens).unwrap();

assert!(rest.is_empty());
assert_eq!(
    parsed,
    Op::Addition(Box::new((
        Op::Number(10),
        Op::Subtraction(Box::new((
            Op::Number(3),
            Op::Number(4),
        ))),
    ))),
)

Macros

You can implement nom::Parser for your token type with the token_parser macro:

#
logos_nom_bridge::token_parser!(token: Token);

If some enum variants of your token type contain data, you can implement a nom::Parser for them using the data_variant_parser macro:

#
#[derive(Clone, Debug, PartialEq, Eq, logos::Logos)]
enum Token {
    #[regex(r"-?[0-9]+", |lex| lex.slice().parse())]
    Number(i64),

    // etc.
}

logos_nom_bridge::data_variant_parser! {
    fn parse_number(input) -> Result<Op>;
    pattern = Token::Number(n) => Op::Number(n);
}