#json #byte-stream #low-memory #checker #automata #json-checker #pushdown

bin+lib oxidized-json-checker

A pushdown automaton low memory JSON bytes stream checker

5 unstable releases

0.3.2 May 19, 2020
0.3.1 May 17, 2020
0.3.0 May 17, 2020
0.2.0 May 16, 2020
0.1.0 May 16, 2020

#32 in #low-memory

Custom license

33KB
679 lines

oxidized-json-checker

This is a pure Rust version of the JSON_checker library.

This is a Pushdown Automaton that very quickly determines if a JSON text is syntactically correct. It could be used to filter inputs to a system, or to verify that the outputs of a system are syntactically correct.

You can use it with the std::io::Read Rust trait to checked if a JSON is valid without having to keep it in memory.


lib.rs:

oxidized-json-checker is a library that provides JSON validation without keeping the stream of bytes in memory, it streams the bytes and validate it on the fly using a pushdown automaton.

The original library has been retrieved from json.org and improved to accept every valid JSON element has a valid JSOn document.

Therefore this library accepts a single string or single integer as a valid JSON document, this way we follow the serde_json rules.

Example: validate some bytes

This example shows how you can give the library a simple slice of bytes and validate that it is a valid JSON document.

let text = r#"["I", "am", "a", "valid", "JSON", "array"]"#;
let bytes = text.as_bytes();

oxidized_json_checker::validate(bytes)?;

Example: validate a stream of bytes

This example shows that you can use any type that implements io::Read to the JsonChecker and validate that it is valid JSON.

let stream = streaming_from_the_web()?;

oxidized_json_checker::validate(stream)?;

Example: complex compositions

This example show how you can use the JsonChecker type to check a compressed stream of bytes.

You can decompress the stream, check it using the JsonChecker, and compress it again to pipe it elsewhere. All of that without much memory impact.

use std::io;
use oxidized_json_checker::JsonChecker;

let stdin = io::stdin();
let stdout = io::stdout();

// Wrap the stdin reader in a Snappy reader
// then wrap it in a JsonChecker reader.
let rdr = snap::read::FrameDecoder::new(stdin.lock());
let mut rdr = JsonChecker::new(rdr);

// Wrap the stdout writer in a Snappy writer.
let mut wtr = snap::write::FrameEncoder::new(stdout.lock());

// The copy function will return any io error thrown by any of the reader,
// the JsonChecker throw errors when invalid JSON is encountered.
io::copy(&mut rdr, &mut wtr)?;

// We must check that the final bytes were valid.
rdr.finish()?;

No runtime deps