8 releases
new 0.2.0 | Mar 16, 2023 |
---|---|
0.1.8 | Dec 20, 2022 |
0.1.7 | Nov 9, 2022 |
0.1.6 | Sep 16, 2022 |
0.1.5 | Aug 31, 2022 |
#31 in Science
59 downloads per month
175KB
4K
SLoC
sci-rs
Pure Rust translation of scipy for reliable cross-platform and cross-tier behavior. See the sibling repo, https://github.com/qsib-cbie/sciprs, for pythonic interaction with this library. sci-rs
will prefer idiomatic Rust with feature parity to the scipy interface when possible.
Memory usage is a priority. While use_std
is a default feature, the library will prefer implementations that do not require runtime allocations, unless noted otherwise.
Nightly Feature Instability
With no future fixes in sight, all nightly features will be hidden behind a maybe broken feature unstable
. This breaks much of the const generic implementations that support no_std
use, but at least dyn features will be available on stable.
https://github.com/rust-lang/rust/issues/106423 has been glaciered and there isn't anyone working on fixing const-generic-exprs right now. There isn't anyone available on that working group to mentor new people to work on them either.
There will be some backtracking to move sci-rs
to stable until there is a more clear path forward for Rust support.
What does work
Butterworth bandpass filter design to SOS or BA. SOS filtering with sosfilt and sosfiltfilt. Statistics like standard deviation or median.
use sci_rs::signal::filter::{design::*, sosfiltfilt_dyn};
let filter = iirfilter_dyn::<f32>(
4,
vec![10., 50.],
None,
None,
Some(FilterBandType::Bandpass),
Some(FilterType::Butterworth),
Some(false),
Some(FilterOutputType::Sos),
Some(1666.),
);
let DigitalFilter::Sos(sos) = filter else { panic!("Not SOS filter") };
let data = (0..100000).map(|i| i as f32);
let filtered: Vec<f32> = sosfiltfilt_dyn(data, &sos.sos);
Some old docs with some things that may work
Python :: Replace scipy with sciprs
Here is an example of python that uses sosfiltfilt to bandpass a numpy array
import numpy as np
# from scipy.signal import butter, sosfiltfilt
from sciprs.signal import butter, sosfiltfilt
raw = np.array([0.0, 0.09414586007215595, 0.18745540640340155, ...])
sos = butter(4, [10, 50], btype='bandpass', output='sos', fs=1666)
np_ndarray_filtered_by_rust_code = sosfiltfilt(buttersos, raw, 0)
Rust :: Conventions
Functions that end with .*_st
do not require allocation. Functions that end with .*_dyn
require allocation. Static implementations are preferred for maximizing the no_std
feature set, but some algorithms require allocation and nightly compiler featuers are brittle. An example would be sosfilt_st
and sosfilt_dyn
, where benchmarks show negligible performance changes between static and dynamic implementations and 8.6x performance improvement over scipy.signal.sosfilt
with the same data and parameters.
We use nightly with features generic_const_exprs
and generic_args_infer
to allow static functions to compute intermediate and output sizing when possible. As these features are not complete or stabilized, the .*_dyn
implementations can be considered more stable than .*_st
.
Rust Goal :: Copy paste python with macros
In order to minimize friction from using scipy python code directly in rust, sci-rs will include a macro system for python-compatible syntax to expand the sci-rs side logic that is currently interacting with pyo3 and the local python runtime. The following doesn't exist but provides a vision for the future compatibility.
let filtered_data: nalgebra::DVector<f64> = sciprs! {
raw = np.array([0.0, 0.09414586007215595, 0.18745540640340155, 0.27909975437050305, 0.3682648115914595])
sos = butter(4, [10, 50], btype='bandpass', output='sos', fs=1666)
sosfiltfilt(sos, raw, 0)
};
Build
Install the nightly toolchain, rustup toolchain install nightly
, rust-toolchain.toml
asks for configures the toolchain choice.
Nightly builds have been broken since Nov, 2022, see https://github.com/rust-lang/rust/issues/106423. Default features compile on stable.
By default, sccache
is enabled to match CI builds and improve build from clean speeds for some of the heavier dependent crates. Install with cargo install sccache --no-default-features
.
In general, cargo build
, cargo test
, and cargo doc
work like usual. sci-rs
does not bind against your local python installation like sciprs
will. The CI workflow runs the unit tests via cargo nextest run
to disallow slow unit tests and to facilitate JUnit + GitHub integration.
When building ndarray or nalgebra, you should not need LAPACK or blas, but eventually it will be an option to accelerate usage on systems that have the option.
System Linear Algebra Dependency
I haven't decided if we will be primarily using ndarray or nalgebra, for now both are supported. Either going back to python can be converted to a numpy ndarray with the python runtime. Between the two crates, one of the primary reasons for creating sci-rs
is to support targets that may not have reasonable access to LAPACK or blas. The choice of library will be determined on feature compatibility rather than performance. That said, there will be many instances where the performance is great, and there will be benchmarks.
ndarray and nalgebra both have features related to matrixmultiply and lapack.
Iterator patterns are used where possible for better compatibility. In the future, rayon may be on the radar for parallelizing compute.
Compare
To compare functionality between sci-rs, sciprs, and scipy, some tests include values that were manually extracted for comparison. The versions are frozen in requirements.txt
. Along with the virtualenv usage for sciprs
, It is recommended to also develop sci-rs
against a venv with frozen versions.
- Creating a venv
python3 -m venv venv
creates a venv folder that is already gitignore'd
- Activating a venv
source venv/bin/activate
- Installing the frozen package versions for development
pip3 install -r requirements.txt
- Deactivating the venv
deactivate
Test
For correctness, results from scipy are directly compared with sciprs output. At the moment, this requires manual scripting and testing. A local python virtualenv is recommended for development of this crate.
Gnuplot
Tests may be used to explore, so gnuplot
is a required dev dependency. You can find more info at http://www.gnuplot.info
- macOS
brew install gnuplot
- linux
apt install gnuplot
Use println!("x = {:?}", x);
to print out floating point data in a format that is copy-pastable into python:
sin_wave = [0.0, 0.5877852522924731, 0.9510565162951535, 0.9510565162951536, 0.5877852522924732, 1.2246467991473532e-16, -0.587785252292473, -0.9510565162951535, -0.9510565162951536, -0.5877852522924734]
Use gnuplot
to plot your data directly in the terminal or in a separate window
let mut fig = Figure::new();
fig.axes2d().lines(
correlations
.iter()
.enumerate()
.map(|(i, _)| i)
.collect::<Vec<_>>(),
correlations,
&[Caption("Sin Wave")],
);
// Comment out the dumb terminal setting to make a test interactive!
fig.set_pre_commands("set term dumb 100 40");
fig.show().unwrap();
1 +-----------------------------------------------------------------------------------+
| ************ |
| ** ** Sin Wave ******* |
0.8 |+ *** ** +|
| ** ** |
0.6 |+ ** ** +|
| * * |
| * ** |
0.4 |+ * * +|
| ** * |
0.2 |+ * * +|
| * ** |
|* * |
0 |+ * +|
| * |
| * |
-0.2 |+ * +|
| ** |
-0.4 |+ * +|
| * |
| * |
-0.6 |+ ** *|
| ** ** |
-0.8 |+ ** ** +|
| ** ** |
| ************* |
-1 +-----------------------------------------------------------------------------------+
0 1 2 3 4 5 6 7 8 9
Maturin + Scipy + Sciprs + Numpy + Matplotlib
Once a method is implemented locally, your local sciprs
crate can point to its dependency at your local sci-rs
directory sci-rs = { path = "../sci-rs" }
. You can maturin develop
to build and install changes made in Rust and script comparisons directly through python.
(sciprs-env) jwtrueb@macbook-pro sciprs % maturin develop --release
🔗 Found pyo3 bindings
🐍 Found CPython 3.9 at /Users/jwtrueb/Desktop/workspace/sciprs/sciprs-env/bin/python
Compiling autocfg v1.1.0
Compiling target-lexicon v0.12.4
...
Compiling nalgebra v0.31.1
Compiling numpy v0.16.2
Compiling sci-rs v0.1.2 (/Users/jwtrueb/Desktop/workspace/sci-rs/sci-rs)
Compiling nalgebra-numpy v0.3.0 (https://github.com/qsib-cbie/nalgebra-numpy#427a8135)
Compiling sciprs v0.1.0 (/Users/jwtrueb/Desktop/workspace/sciprs)
Finished release [optimized] target(s) in 12.10s
📦 Built wheel for CPython 3.9 to /var/folders/n_/s24_t4gj6rnf7r2kxj376df40000gn/T/.tmphMfGAv/sciprs-0.1.0-cp39-cp39-macosx_11_0_arm64.whl
importlib.reload doesn't work, so you need to relaunch the interpreter each time.
# load the module with the new Rust code
import sciprs
# do what you want
print(sciprs.call_new_function(1,2,3))
Dependencies
~4–7MB
~136K SLoC