#rise #data-compression #floating-point #datasets #testdata

app sqsh-testdata

Tool to generate testdata for data compression

3 releases

Uses new Rust 2021

0.1.2 Aug 2, 2022
0.1.1 Aug 2, 2022
0.1.0 Jul 31, 2022
Download history 78/week @ 2022-07-31 12/week @ 2022-08-07

90 downloads per month

MIT/Apache

9KB
184 lines

Sqsh Testdata Generator

A tool to generate synthetic floating point datasets. Currently the only datasets with a gaussian/normal distribution are supported. Further distributions can be added if the need may rise.

Install

Use stable cargo release to install the toolset:

cargo install sqsh-testdata

Use latest version from github:

git clone https://github.com/sqsh-project/testdata.git && \
cd sqsh-testdata && \
cargo install --path .

Usage

The easiest way to generate a dataset is to specify mean, standard deviation and the number of numbers to be generated via num:

sqsh-testdata --mean 10 --std 2 --size 1000 # mandatory arguments

The default generates single-precision floating-point data. This can be changed using the --datatype argument:

sqsh-testdata --mean 10 --std 2 --size 1000 --datatype double

Should it be necessary to define a different endianess than the machine native the --endianess argument can be used

sqsh-testdata --mean 10 --std 2 --size 1000 --endianess little

By providing a --seed value the generated data can be reproduced on different environments:

sqsh-testdata --mean 10 --std 2 --size 1000 --seed 42

The data can be output to a file by piping the result to a file:

sqsh-testdata --mean 10 --std 2 --size 1000 > /tmp/data.raw  # save to file

For more information the help menu can be used: td --help

Dependencies

~4MB
~74K SLoC