#json

streamson-lib

Library for processing large JSONs

13 releases (6 major breaking)

6.0.0 Oct 12, 2020
5.0.2 Oct 10, 2020
4.1.0 Aug 29, 2020
4.0.0 Jul 28, 2020
0.2.0 May 29, 2020

#29 in Parser tooling

Download history 73/week @ 2020-06-30 66/week @ 2020-07-07 22/week @ 2020-07-14 47/week @ 2020-07-21 72/week @ 2020-07-28 49/week @ 2020-08-04 35/week @ 2020-08-11 12/week @ 2020-08-18 41/week @ 2020-08-25 65/week @ 2020-09-01 20/week @ 2020-09-08 3/week @ 2020-09-15 4/week @ 2020-09-22 27/week @ 2020-09-29 92/week @ 2020-10-06 42/week @ 2020-10-13

154 downloads per month
Used in 5 crates

MIT license

135KB
2.5K SLoC

docs.rs

Streamson Lib

Rust library to handle large JSONs.

Note that it doesn't fully validates whether the input JSON is valid. This means that invalid JSONs might pass without an error.

Trigger strategy

It doesn't actually perform parses json into data. It just takes JSON parts and triggers handlers when a path is matched.

Filter strategy

It actually alters the JSON. If the path is matched the matched part should be removed from output json.

Extract strategy

Only extracts matched data, nothing else.

Convert strategy

Alters the JSON by calling convert functions to matched parts.

Examples

Trigger

use streamson_lib::{strategy, GenericError, PrintLn, Simple};

let mut trigger = strategy::Trigger::new();
let handler = Arc::new(Mutex::new(PrintLn::new());
let matcher = Simple(r#"{"users"}[]"#).unwrap();
trigger.add_matcher(Box::new(matcher), &[handler]);

let mut buffer = [0; 2048];
while let Ok(size) = input.read(&mut buffer[..]) {
	filter.process(&buffer[..size])?;
}

Filter

use streamson_lib::{strategy, error::GenericError, matcher::Simple};

let mut filter = strategy::Filter::new();
let matcher = Simple(r#"{"users"}[]"#).unwrap();
filter.add_matcher(Box::new(matcher), &[handler]);

let mut buffer = [0; 2048];
while let Ok(size) = input.read(&mut buffer[..]) {
	let output_data = filter.process(&buffer[..size])?;
}

Extract

use streamson_lib::{strategy, error::GenericError, matcher::Simple};

let mut extract = strategy::Extract::new();
let matcher = Simple(r#"{"users"}[]"#).unwrap();
extract.add_matcher(Box::new(matcher), &[handler]);

let mut buffer = [0; 2048];
while let Ok(size) = input.read(&mut buffer[..]) {
	let output_data = extract.process(&buffer[..size])?;
}

Convert

use streamson_lib::{strategy, matcher, handler};

let mut convert = strategy::Convert::new();
let matcher = matcher::Simple::new(r#"{"list"}[]"#).unwrap();

convert.add_matcher(
	Box::new(matcher),
	Box::new(|_, _| vec![b'"', b'.', b'.', b'.', b'"'])
);

let mut buffer = [0; 2048];
while let Ok(size) = input.read(&mut buffer[..]) {
	let output_data = extract.process(&buffer[..size])?;
}

Traits

Custom Handlers

You can define your custom handler.

use streamson_lib::handler;

...

struct CustomHandler;

impl handler::Handler for CustomHandler {
	...
}

Custom Matchers

You can define custom matchers as well.

use streamson_lib::matcher;

struct CustomMatcher;

impl matcher::MatchMaker for CustomMatcher {
	...
}

No runtime deps