#json

app streamson-bin

A program which processes large JSON data

13 releases (6 major breaking)

new 6.0.0 Oct 12, 2020
5.0.2 Oct 10, 2020
4.1.0 Aug 29, 2020
4.0.0 Jul 28, 2020
0.2.0 May 29, 2020

#64 in Parser tooling

Download history 36/week @ 2020-06-27 8/week @ 2020-07-04 19/week @ 2020-07-11 28/week @ 2020-07-18 22/week @ 2020-07-25 10/week @ 2020-08-01 8/week @ 2020-08-08 25/week @ 2020-08-15 2/week @ 2020-08-22 21/week @ 2020-08-29 29/week @ 2020-09-05 2/week @ 2020-09-12 2/week @ 2020-09-19 2/week @ 2020-09-26 16/week @ 2020-10-03 39/week @ 2020-10-10

66 downloads per month

MIT license

155KB
3K SLoC

Streamson Bin

Splits stdout (in JSON format) according to given options. It is supposed to be memory efficient and fast to process.

Examples

Consider following context of input.json file:

{
	"users": [
		{"name": "user1", "groups": ["admins", "staff"], "password": "secret1"},
		{"name": "user2", "groups": ["staff"], "password": "secret2"}
	]
}

Split each user into separate JSON

cat input.json | streamson-bin extract --depth 2

Output:

{"name": "user1", "authors": ["admins", "staff"], "password": "secret1"}{"name": "user2", "authors": ["staff"], "password": "secret2"}

Mask password

cat input.json | \
	streamson-bin extract --depth 2 | \
	streamson-bin convert --replace '"***"' --simple '{"password"}'

Output:

{"name": "user1", "groups": ["admins", "staff"], "password": "***"}{"name": "user2", "groups": ["staff"], "password": "***"}

Remove groups

cat input.json | \
	streamson-bin extract --depth 2 | \
	streamson-bin convert --replace '"***"' --simple '{"password"}' | \
	streamson-bin filter --simple '{"groups"}'

Output:

{"name": "user1", "password": "***"}{"name": "user2", "password": "***"}

Store names of users into a separate files

cat input.json | \
	streamson-bin extract --depth 2 | \
	streamson-bin convert --replace '"***"' --simple '{"password"}' | \
	streamson-bin filter --simple '{"groups"}' | \
	streamson-bin trigger --file simple '{"name"}' names.out

Output:

{"name": "user1", "password": "***"}{"name": "user2", "password": "***"}

names.out:

"user1"
"user2"

Dependencies

~735KB
~11K SLoC