#csv #row #billion #challenge #generate #weather #station

app billion-row-gen

Generates a large number of rows for the one billion row challenge

1 unstable release

0.1.0 Feb 27, 2024

#196 in Profiling

MIT license

445KB
127 lines

Rust One Billion Row Challenge Generator

Generates one billion rows of data for the 1BRC

https://github.com/gunnarmorling/1brc

Installation

cargo install billion-row-gen

Usage

Defaults:

  • The row count defaults to 1,000,000,000.
  • The tool will use the ./data/weather_stations.csv file to generate the weather station names.
  • The output will be stored in ./data/measurements.txt.

All of these options may be configured:

billion-row-gen \
    --rows 1000 \
    --weather-stations ./data/weather.csv \
    --output ./data/out.txt

Dependencies

~9–19MB
~234K SLoC