9 releases (stable)
1.3.2 | Oct 16, 2022 |
---|---|
1.3.1 | Jul 17, 2022 |
1.2.1 | Dec 5, 2021 |
1.2.0 | Mar 5, 2021 |
0.1.1 | Jul 11, 2018 |
#849 in Command line utilities
28 downloads per month
14KB
208 lines
xstream
A command line tool to split a stream by a delimiter and pipe each section to a child process.
Installation
cargo install xstream-util
Benchmarks
For a simple illustration of the speed up for reasonably sized streams, the following simple benchmark compares generating 1001 streams of integers and summing them with bc
.
First, generate a null delimited set of streams with
time for I in {10000..11000}; do seq $I; echo -ne '\0'; done
This stream is roughly 50M, making each stream roughly 50k.
I then piped this into xstream
as
| time xstream -0 -- bash -c 'paste -sd+ | bc' > /dev/null
and xargs
as
| time xargs -0I@ bash -c '<<< "@" head -n-1 | paste -sd+ | bc' > /dev/null
which on my system gives:
Program | User | System | Elapsed |
---|---|---|---|
xstream |
10.21s | 1.67s | 0:09.58 |
xargs |
15.72s | 2.85s | 0:14.52 |
This benchmark is a toy example, but xstream
already provides a 30% speed up when each stream is only 50k.
Dependencies
~0–5MB
~83K SLoC