3 releases (breaking)
0.3.0 | Nov 14, 2024 |
---|---|
0.2.14 |
|
0.2.12 |
|
0.1.5 | Mar 27, 2024 |
#1427 in Parser implementations
157 downloads per month
105KB
275 lines
Convert CSV To Parquet (CC2P)
(CC2P) is a Rust-based project that converts CSV files in a selected folder into parquet format. This tool provides a simple and efficient way of handling and converting your CSV data files.
Installation & Usage
Prerequisites
- Rust 1.75
Building
Provide instructions on how to build the project, for example, installing the Rust compiler and necessary crates.
Here is how to install the cc2p
directly from the Git repository:
cargo install cc2p
Running
Provide Instructions on how to run the scripts. For example, how to specify the input CSV file and the output Parquet file.
cc2p [OPTIONS] /path/to/csv/file.csv
Options:
- delimiter : delimiter char used in CSV files (default:
,
) - no-header : whether to include the header in the CSV search column (default:
false
) - worker: Number of worker threads to use for performing the task (default:
4
) - sampling: Number of rows to sample for inferring the schema (default:
100
)
> cc2p --help
Convert a CSV to parquet file format
Usage: cc2p.exe [OPTIONS] [PATH]
Arguments:
[PATH] Represents the folder path for CSV search [default: *.csv]
Options:
-d, --delimiter <DELIMITER> Represents the delimiter used in CSV files [default: ,]
-n, --no-header Represents whether to include the header in the CSV search column
-w, --worker <WORKER> Number of worker threads to use for performing the task [default: 1]
-s, --sampling <SAMPLING> Number of rows to sample for inferring the schema. [default: 100]
-h, --help Print help
-V, --version Print version
Features
- Fast and reliable CSV to Parquet conversion.
- Multithreaded processing with the help of the tokio crate.
- Progress indication during conversion with the help of the indicatif crate.
Contributing
If you wish to contribute, please feel free to fork the repository, make your changes, and submit a pull request. All contributions are welcome!
License
This project is licensed under MIT, see the LICENSE file for details.
Contact
Project Link: https://github.com/rayyildiz/cc2p
Dependencies
~31–43MB
~812K SLoC