#postgresql #parquet #command-line-tool #data-processing

app postquet

PostQuet: Stream PostgreSQL tables/queries to Parquet files seamlessly with this high-performance, Rust-based command-line tool

1 unstable release

0.1.0 Apr 9, 2023

#44 in #parquet

Apache-2.0

61KB
195 lines

Build and Test

STATUS: IN DEVELOPMENT

logo

PostQuet: Streaming PostgreSQL to Parquet Exporter

PostQuet is a powerful and efficient command-line tool written in Rust that enables you to stream PostgreSQL tables to Parquet files seamlessly. Designed for data engineers, analysts, and developers, this tool simplifies the process of converting PostgreSQL data into the widely used columnar storage format, Parquet.

Key Features

  • Streaming data conversion: PostQuet allows for the streaming of data between PostgreSQL and Parquet, reducing memory overhead and enabling the processing of large datasets.
  • High-performance: Leveraging Rust's speed and safety, PostQuet offers excellent performance while ensuring the reliability of your data conversion.
  • Asynchronous processing: The tool utilizes asynchronous processing techniques to optimize efficiency and minimize the time spent on data conversion tasks.
  • Support for complex data types and table structures: PostQuet handles a wide range of data types and table structures, ensuring compatibility with diverse PostgreSQL databases.
  • User-friendly command-line interface: With an intuitive CLI, PostQuet makes it easy for users to export PostgreSQL tables to Parquet files, regardless of their technical expertise.

With PostQuet, you can quickly and easily convert your PostgreSQL data into the Parquet format, unlocking new possibilities for data analysis and processing.

Dependencies

~34–48MB
~1M SLoC