43 releases (28 major breaking)

55.0.0 Apr 11, 2025
54.3.1 Mar 30, 2025
54.2.1 Feb 27, 2025
54.0.0 Dec 23, 2024
27.0.0 Nov 14, 2022

#226 in Encoding

Download history 187297/week @ 2025-01-01 279379/week @ 2025-01-08 233751/week @ 2025-01-15 243148/week @ 2025-01-22 237350/week @ 2025-01-29 270836/week @ 2025-02-05 337554/week @ 2025-02-12 416833/week @ 2025-02-19 538173/week @ 2025-02-26 496029/week @ 2025-03-05 478927/week @ 2025-03-12 460000/week @ 2025-03-19 452831/week @ 2025-03-26 487785/week @ 2025-04-02 483713/week @ 2025-04-09 411911/week @ 2025-04-16

1,922,884 downloads per month
Used in 54 crates (18 directly)

Apache-2.0

3.5MB
65K SLoC

Transfer data between the Arrow memory format and JSON line-delimited records.

See the module level documentation for the reader and writer for usage examples.

Binary Data

As per RFC7159 JSON cannot encode arbitrary binary data. A common approach to workaround this is to use a binary-to-text encoding scheme, such as base64, to encode the input data and then decode it on output.

#
// The data we want to write
let input = BinaryArray::from(vec![b"\xDE\x00\xFF".as_ref()]);

// Base64 encode it to a string
let encoded: StringArray = b64_encode(&BASE64_STANDARD, &input);

// Write the StringArray to JSON
let batch = RecordBatch::try_from_iter([("col", Arc::new(encoded) as _)]).unwrap();
let mut buf = Vec::with_capacity(1024);
let mut writer = LineDelimitedWriter::new(&mut buf);
writer.write(&batch).unwrap();
writer.finish().unwrap();

// Read the JSON data
let cursor = Cursor::new(buf);
let mut reader = ReaderBuilder::new(batch.schema()).build(cursor).unwrap();
let batch = reader.next().unwrap().unwrap();

// Reverse the base64 encoding
let col: BinaryArray = batch.column(0).as_string::<i32>().clone().into();
let output = b64_decode(&BASE64_STANDARD, &col).unwrap();

assert_eq!(input, output);

Dependencies

~8.5MB
~147K SLoC