#miden #source #component #scanner #token #parser #peek


Reusable parsing components for use in Miden projects

1 unstable release

0.1.0 Jul 12, 2023

#765 in Programming languages

Download history 11/week @ 2024-01-11 2/week @ 2024-01-18 8/week @ 2024-02-08 14/week @ 2024-02-15 25/week @ 2024-02-22 23/week @ 2024-02-29 23/week @ 2024-03-07 21/week @ 2024-03-14 19/week @ 2024-03-21 43/week @ 2024-03-28 23/week @ 2024-04-04 22/week @ 2024-04-11 23/week @ 2024-04-18 12/week @ 2024-04-25

93 downloads per month
Used in 5 crates (via air-parser)

MIT license

382 lines


This crate provides additional infrastructure on top of miden-diagnostics for parsing within a compiler frontend.


The following are features provided by this crate:

  • An abstraction called Source that provides the core operations required by the scanner component of a parser, along with a "standard" implementation called FileMapSource which builds on the SourceFile type provided by miden-diagnostics
  • Scanner, a low-level component intended for consumption from a lexer. At a high level, the scanner operates on the input source to allow a lexer to precisely control what characters in the source are part of the current token, the ability to get a slice associated with the current token, as well as the ability to control advancement of the underlying source stream (i.e. get the current char, pop the current char, peek the next char, etc).
  • Parse<T> a trait which describes what is needed to parse a T, and exposes the API for doing so
  • Parser<C> represents a parser with a configuration type C that is capable of parsing any compatible Parse implementation. This parser is designed for use with components provided by miden-diagnostics, specifically CodeMap and DiagnosticsHandler. It abstracts away some of the boilerplate involved with common parsing tasks, e.g. parse_file and parse_string.

This crate is known to work well with LALRPOP, and only requires that a compiler frontend provide a lexer and token type specific to the language being parsed. See the examples folder for a complete end-to-end demonstration of how to use this crate (and miden-diagnostics) with LALRPOP for a simple language.


~87K SLoC