9 releases (stable)

1.3.2 Oct 16, 2022
1.3.1 Jul 17, 2022
1.2.1 Dec 5, 2021
1.2.0 Mar 5, 2021
0.1.1 Jul 11, 2018

#849 in Command line utilities

28 downloads per month

MIT license

14KB
208 lines

xstream

crates.io documentation license tests

A command line tool to split a stream by a delimiter and pipe each section to a child process.

Installation

cargo install xstream-util

Benchmarks

For a simple illustration of the speed up for reasonably sized streams, the following simple benchmark compares generating 1001 streams of integers and summing them with bc.

First, generate a null delimited set of streams with

time for I in {10000..11000}; do seq $I; echo -ne '\0'; done

This stream is roughly 50M, making each stream roughly 50k.

I then piped this into xstream as

| time xstream -0 -- bash -c 'paste -sd+ | bc' > /dev/null

and xargs as

| time xargs -0I@ bash -c '<<< "@" head -n-1 | paste -sd+ | bc' > /dev/null

which on my system gives:

Program User System Elapsed
xstream 10.21s 1.67s 0:09.58
xargs 15.72s 2.85s 0:14.52

This benchmark is a toy example, but xstream already provides a 30% speed up when each stream is only 50k.

Dependencies

~0–5MB
~83K SLoC