#serde #proc-macro #tokenstream

serde_tokenstream

A serde deserializer for proc-macro TokenStreams

4 releases

0.1.3 Jan 12, 2022
0.1.2 Aug 12, 2020
0.1.1 Aug 8, 2020
0.1.0 May 15, 2020

#16 in Procedural macros

Download history 1538/week @ 2021-10-02 1494/week @ 2021-10-09 1398/week @ 2021-10-16 1349/week @ 2021-10-23 1296/week @ 2021-10-30 1716/week @ 2021-11-06 2001/week @ 2021-11-13 1676/week @ 2021-11-20 2231/week @ 2021-11-27 1561/week @ 2021-12-04 1703/week @ 2021-12-11 1228/week @ 2021-12-18 312/week @ 2021-12-25 1112/week @ 2022-01-01 1629/week @ 2022-01-08 1419/week @ 2022-01-15

4,575 downloads per month
Used in 17 crates (6 directly)

Apache-2.0

54KB
1.5K SLoC

serde_tokenstream

This Rust crate is intended for use with macros that need bespoke configuration. It's implemented as a serde::Deserializer that operates on a proc_macro2::TokenSteam (easily converted from the standard proc_macro::TokenStream).

Usage

Say we're building a custom proc macro that you want consumers to use like this:

#[MyMacro {
    name = "SNPP",
    owner = "Canary M Burns",
    details = {
        kind = Fission,
        year_of_opening = 1968,
    }
}]
fn some_func() {
    ...
}

The function that implements the proc macro must have two parameters (both of type proc_macro::TokenStream): attributes (the tokens with the braces that follow the name of the macro), and the item (the function, type, etc. to which the macro is applied):

#[proc_macro_attribute]
pub fn MyMacro(
    attr: proc_macro::TokenStream,
    item: proc_macro::TokenStream,
) -> proc_macro::TokenStream {
    ...
}

We'll first define the struct type that represents the configuration and derive a serde::Deserialize:

#[derive(Deserialize)]
struct Config {
    name: String,
    owner: String,
    details: ConfigDetails,
}

#[derive(Deserialize)]
struct ConfigDetails {
    kind: ConfigDetailsType,
    year_of_opening: usize,
}

#[derive(Deserialize)]
enum ConfigDetailsType {
    Coal,
    Fission,
    Hydroelectric,
}

Now we can parse attr into the Config struct with serde_tokenstream::from_tokenstream:

use proc_macro2::TokenStream;
use serde_tokenstream::from_tokenstream;

#[allow(non_snake_case)]
#[proc_macro_attribute]
pub fn MyMacro(
    attr: proc_macro::TokenStream,
    item: proc_macro::TokenStream,
) -> proc_macro::TokenStream {
    let config = match from_tokenstream::<Config>(&TokenStream::from(attr)) {
        Ok(c) => c,
        Err(err) => return err.to_compile_error().into(),
    };

    ...
}

See the serde documentation for the full range of controls that can be applied to types and their members.

TokenStream and syn::* values

In some cases, it's useful to pass TokenStream values as parameters to a macro. In this case we can use the TokenStreamWrapper which is a wrapper around TokenStream that implements Deserialize or ParseWrapper which is a wrapper around syn::Parse that implements Deserialize. The latter is useful for passing in, for example, a syn::Path, or other specific entities from the syn crate.

Error Handling

Note that errors will highlight the problematic portion of consuming code:

#[MyMacro{
    name = "Rocinante",
    owner = "Rocicorp",
    details = {
        kind = Fusion,
        year_of_opening = 2347
    }
}]
fn deploy() {
    ...
}
error: unknown variant `Fusion`, expected one of `Coal`, `Fission`, `Hydroelectric`
 --> tests/test_err1.rs:7:16
  |
7 |         kind = Fusion,
  |                ^^^^^^

Dependencies

~0.4–1MB
~23K SLoC

~a