#parquet #thrift #reader-writer #read #byte #write #sync-async

parquet-format-safe

Safe Parquet and Thrift reader and writer (sync and async)

6 releases

0.2.4 Aug 17, 2022
0.2.3 Aug 13, 2022
0.1.0 Aug 10, 2022

#241 in Encoding

Download history 14384/week @ 2023-11-23 14677/week @ 2023-11-30 13068/week @ 2023-12-07 14082/week @ 2023-12-14 9116/week @ 2023-12-21 10002/week @ 2023-12-28 13355/week @ 2024-01-04 14886/week @ 2024-01-11 14190/week @ 2024-01-18 14030/week @ 2024-01-25 14586/week @ 2024-02-01 13463/week @ 2024-02-08 15039/week @ 2024-02-15 14863/week @ 2024-02-22 17246/week @ 2024-02-29 11392/week @ 2024-03-07

61,195 downloads per month
Used in 69 crates (2 directly)

MIT/Apache

495KB
11K SLoC

parquet-format-safe

This crate contains an implementation of Thirft and generated Rust code associated to Parquet's thrift definition.

Note that this crate does not contain functionality to read the parquet format. See parquet2 for such functionality.

  • supports sync and async read API
  • supports sync and async write API
  • the write API returns the number of written bytes
  • the read API is panic free
  • the read API has a bound on the maximum number of possible bytes read, to avoid OOM.

It must be used with the fork of thrift's compiler available at https://github.com/jorgecarleitao/thrift/tree/safe .

Dependencies

~0–255KB