#commit

app streambed-logged-cli

A CLI for a file-system-based commit log

4 releases

0.11.1 Sep 13, 2024
0.11.0 Aug 16, 2024
0.10.2 Aug 13, 2024
0.10.1 Aug 13, 2024

#44 in #commit

Download history 389/week @ 2024-08-12 8/week @ 2024-08-19 2/week @ 2024-08-26 157/week @ 2024-09-09 24/week @ 2024-09-16 35/week @ 2024-09-30 2/week @ 2024-10-07

70 downloads per month

Apache-2.0

150KB
3K SLoC

logged

The logged command provides a utility for conveniently operating on file-based commit logs. Functions such as the ability to consume a JSON file of records, or produce them, are available. No assumptions are made regarding the structure of a record's value (payload), or whether it is encrypted or not. The expectation is that a separate tool for that concern is used in a pipeline if required.

Running with an example write followed by a read

First get the executable:

cargo install streambed-logged-cli

...or build it from this repo:

cargo build --bin logged --release

...and make a build available on the PATH (only for cargo build and just once per shell session):

export PATH="$PWD/target/release":$PATH

...then write some data to a topic named my-topic:

echo '{"topic":"my-topic","headers":[],"key":0,"value":"SGkgdGhlcmU=","partition":0}' | \
  logged --root-path=/tmp produce --file -

...then read it back:

logged --root-path=/tmp subscribe --subscription my-topic --idle-timeout=0ms

...which would output:

{"topic":"my-topic","headers":[],"timestamp":null,"key":0,"value":"SGkgdGhlcmU=","partition":0,"offset":0}

If you're using nushell then you can do nice things like base64 decode payload values along the way:

logged --root-path=/tmp subscribe --subscription my-topic --idle-timeout=0m | from json --objects | update value {decode base64}

...which would output:

╭────┬──────────┬────────────────┬───────────┬─────┬──────────┬───────────┬────────╮
│  # │  topic   │    headers     │ timestamp │ key │  value   │ partition │ offset │
├────┼──────────┼────────────────┼───────────┼─────┼──────────┼───────────┼────────┤
│  0 │ my-topic │ [list 0 items] │           │   0 │ Hi there │         00 │
╰────┴──────────┴────────────────┴───────────┴─────┴──────────┴───────────┴────────╯

Use --help to discover all of the options.

Dependencies

~10–19MB
~249K SLoC