parquet2lance

29 releases

Uses new Rust 2024

0.5.0	Jan 24, 2025
0.4.4	Jul 30, 2024
0.4.2	Jun 6, 2024
0.4.1	Mar 7, 2024
0.0.3	Mar 29, 2023

#20 in #gcs

1,432 downloads per month

Custom license

38KB
383 lines

Convert parquet files to lance
Working with GCS
- Using gcloud auth application-default login to generate Application Default Credentials (ADC)

parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/ \
  -o test.lance -O

parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/us-states.parquet \
  -o test.lance -O

From GCS to GCS

parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/ \
  -o gs://your-bucket-name/bigquery/us-states/

parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/us-states.parquet \
  -o gs://your-bucket-name/bigquery/us-states/us-states.lance

Working with local FS

gsutil cp gs://cloud-samples-data/bigquery/us-states/us-states.parquet test.parquet

parquet2lance -i test.parquet -o test.lance -O

Dependencies

~86–120MB
~2M SLoC