6 releases

0.3.1 Sep 8, 2020
0.3.0 Sep 5, 2020
0.1.3 May 1, 2020
0.1.2 Dec 30, 2018

#1527 in Parser implementations

MIT/Apache

24KB
505 lines

Basic usage

This is a parser for simple s-expressions written in rust.

There are probably these things you want to do with this lib:

  1. Convert textual representation of a program to simple s-expressions (see section Parsers)
  2. Do some syntactical conversions using s-expressoins (optional)
  3. Execute the program or show the document in some way:
    • interpret simple s-expressions directly
    • convert to a format, which is already executable (llvm, assembly, bytecode, C, html)
    • convert to an other format (AST, typed s-expressions) before doing any of these

Format of parsed content

A simple s-expression consists of tokens, which can either be symbols or lists of other tokens.

A symbol is basically just a string. A list can represent more complicated expressions. There is no representation for different types, but you can just use prefixed symbols ('string:hello', 'int:1') or lists (('string' 'hello'), ('int' '1')). The exact representation and it's meaning can be decided by the user. It's suitable for plain data (instead of json/xml/etc.), interpretable code, converting code into rust at compile time and more.

These expressions are just an intern representation, which can be used for simple structural transformations.

Parsers

The parsing of text is based on reader macros, similar to how they work in Common Lisp.

There are some predefined macro characters for most common use cases like parsing strings, lists and comments, but you can define your own parsers, too (just copy an existing one and modify it). It's reasonable to start parsing using a DelimitedListParser. Just have a look at the example (math.rs) in the tests.

In general, a parser tries to parse symbols until a macro character is encountered. A macro character is bound to a parser, which can be invoked. After a parser is finished, it gives control back to the previous parser to parse the rest of the file.

Unlike the common lisp version, the parser has access to the local list of parsed tokens, which makes it very flexible. This way it's even reasonable to add infix operations to the parser.

The parsed format can even look very different to typical s-expressions.

Dependencies

~2MB
~45K SLoC