1 unstable release
0.1.1 | Feb 14, 2020 |
---|
#35 in #big
13KB
datapan
This is still a test bed. It is not useful
datapan
sifts through enormous files in parallelized Rust to only grab the data you want as quickly and memory-efficiently as possilbe.
Installation
## create/activate venv
# sudo apt-get install python3-venv
# python3 -m venv datapan_env
# source datapan_env/bin/activate
# python -m pip install --upgrade pip
## install datapan
pip install datapan
Usage
import datapan
some_dir = ""
test = datapan.hello_rust(some_dir)
print(test)
Developer Version
- Rust (nightly)
curl https://sh.rustup.rs -sSf | sh
# rustup default nightly
rustup update nightly
- Poetry
pip install poetry
make install
make test
Dependencies
~9MB
~178K SLoC