Convert CSV To Parquet (CC2P)

(CC2P) is a Rust-based project that converts CSV files in a selected folder into parquet format. This tool provides a simple and efficient way of handling and converting your CSV data files.

Installation & Usage

Prerequisites

• Rust 1.85 (edition: 2024)

Building

Provide instructions on how to build the project, for example, installing the Rust compiler and necessary crates.

Here is how to install the cc2p directly from the Git repository:

cargo install cc2p

Running

Provide Instructions on how to run the scripts. For example, how to specify the input CSV file and the output Parquet file.

cc2p [OPTIONS] /path/to/csv/file.csv

Options:

• delimiter : delimiter char used in CSV files (default: ,)
• no-header : whether to include the header in the CSV search column (default: false)
• worker: Number of worker threads to use for performing the task (default: 4)
• sampling: Number of rows to sample for inferring the schema (default: 100)

> cc2p --help

Convert a CSV to parquet file format

Usage: cc2p.exe [OPTIONS] [PATH]

Arguments:
  [PATH]  Represents the folder path for CSV search [default: *.csv]

Options:
  -d, --delimiter <DELIMITER>  Represents the delimiter used in CSV files [default: ,]
  -n, --no-header              Represents whether to include the header in the CSV search column
  -w, --worker <WORKER>        Number of worker threads to use for performing the task [default: 1]
  -s, --sampling <SAMPLING>    Number of rows to sample for inferring the schema. [default: 100]
  -h, --help                   Print help
  -V, --version                Print version

MacOS Users

NOTE for macOS Users: Our Apple signing/notarization is not entirely done yet, thus you have to run the following command once to run the application. Download the app and run this command:

xattr -c cc2p

Features

• Fast and reliable CSV to Parquet conversion.
• Multithreaded processing with the help of the tokio crate.
• Progress indication during conversion with the help of the indicatif crate.

Contributing

If you wish to contribute, please feel free to fork the repository, make your changes, and submit a pull request. All contributions are welcome!

License

This project is licensed under MIT, see the LICENSE file for details.

Contact

Project Link: https://github.com/rayyildiz/cc2p