#token #serde #deserializer #transcoding #sink #future #tokenize

serde_token

Utility for transcoding a Serde deserializer into a sink of tokens

2 releases

0.0.2 Jun 24, 2019
0.0.1 Jun 24, 2019

#1462 in Encoding

MIT license

22KB
573 lines

serde_token

Crates.io Documentation MIT license

serde_token provides a utility for transcoding a Serde deserializer into a sink of Tokens.

Installation

Install from Crates.io:

[dependencies]
serde_token = "0.0.2"

Usage

use futures::{unsync::mpsc::unbounded, Future, Sink, Stream};
use serde_json::Deserializer;
use serde_token::{tokenize, Token};

let mut de = Deserializer::from_str(r#" [ {"a":false}, "hello", 3 ] "#);
let (token_sink, token_stream) = unbounded::<Token>();

tokenize(&mut de, token_sink).unwrap();

let expected = token_stream.collect().wait().unwrap();
assert_eq!(expected, vec![
    Token::Seq { len: None },
    Token::Map { len: None },
    Token::Str("a"),
    Token::Bool(false),
    Token::MapEnd,
    Token::Str("hello"),
    Token::U64(3),
    Token::SeqEnd,
])

TODO

  • look into handling Sink sending better (i.e. support back-pressure)

Changelog

Version Change Summary
v0.0.2 adds README
v0.0.1 initial release

Contributing

  1. Fork it https://github.com/your_username/serde_token/fork
  2. Create your feature branch (git checkout -b feature/fooBar)
  3. Commit your changes (git commit -am 'Add some fooBar')
  4. Push to the branch (git push origin feature/fooBar)
  5. Create a new Pull Request

Maintainers

License

MIT

Dependencies

~175–415KB