#tensor

n5

Rust implementation of the N5 tensor file system format

13 releases (6 breaking)

✓ Uses Rust 2018 edition

new 0.7.1 Jan 19, 2020
0.6.1 Jan 8, 2020
0.5.0 Nov 15, 2019
0.4.0 Jun 6, 2019
0.2.0 Mar 10, 2018

#29 in Filesystem

Download history 5/week @ 2019-10-07 39/week @ 2019-10-14 32/week @ 2019-10-21 40/week @ 2019-10-28 32/week @ 2019-11-04 45/week @ 2019-11-11 35/week @ 2019-11-18 39/week @ 2019-11-25 72/week @ 2019-12-02 329/week @ 2019-12-09 13/week @ 2019-12-16 25/week @ 2019-12-23 34/week @ 2019-12-30 115/week @ 2020-01-06 110/week @ 2020-01-13

307 downloads per month
Used in 3 crates

MIT/Apache

87KB
2K SLoC

N5 Build Status Coverage

A (mostly pure) rust implementation of the N5 "Not HDF5" n-dimensional tensor file system storage format created by the Saalfeld lab at Janelia Research Campus.

Compatible with Java N5 Version 2.1.3.

Differences from Java N5

  • Dataset paths are relative. The root path in a dataset is "", not "/".
  • Dataset paths are more strict. Calling methods with paths outside the dataset, e.g., "..", will return a Result::Err.

Minimum supported Rust version (MSRV)

Stable 1.39

Quick start

[dependencies]
n5 = "0.7"
use n5::prelude::*;
use n5::smallvec::smallvec;

fn n5_roundtrip(root_path: &str) -> std::io::Result<()> {
    let n = N5Filesystem::open_or_create(root_path)?;

    let block_size = smallvec![44, 33, 22];
    let data_attrs = DatasetAttributes::new(
        smallvec![100, 200, 300],
        block_size.clone(),
        DataType::INT16,
        CompressionType::default(),
    );
    let block_data: Vec<i16> = vec![0i16; data_attrs.get_block_num_elements()];

    let block_in = VecDataBlock::new(
        block_size,
        smallvec![0, 0, 0],
        block_data.clone());

    let path_name = "test/dataset/group";

    n.create_dataset(path_name, &data_attrs)?;
    n.write_block(path_name, &data_attrs, &block_in)?;

    let block_out = n.read_block::<i16>(path_name, &data_attrs, smallvec![0, 0, 0])?
        .expect("Block is empty");
    assert_eq!(block_out.get_data(), &block_data);

    Ok(())
}

fn main() {
    n5_roundtrip("tmp.n5").expect("N5 roundtrip failed!");
}

Status

This library is compatible with all N5 datasets the authors have encountered and is used in production services. However, some aspects of the library are still unergonomic and interfaces may still undergo rapid breaking changes.

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Dependencies

~1.2–2.6MB
~54K SLoC