#datasets

nightly pointcloud

An accessor layer for goko

12 releases

new 0.3.9 Sep 17, 2020
0.3.8 Sep 14, 2020
0.3.6 Aug 24, 2020
0.3.5 Jul 23, 2020
0.1.2 Feb 27, 2020

#57 in Science

Download history 11/week @ 2020-06-04 1/week @ 2020-06-11 20/week @ 2020-06-25 13/week @ 2020-07-02 1/week @ 2020-07-09 33/week @ 2020-07-16 40/week @ 2020-07-23 4/week @ 2020-07-30 13/week @ 2020-08-06 13/week @ 2020-08-13 27/week @ 2020-08-20 19/week @ 2020-08-27 36/week @ 2020-09-03 41/week @ 2020-09-10 21/week @ 2020-09-17

91 downloads per month
Used in 2 crates

Custom license

120KB
3K SLoC

Point Cloud

A dataset access layer that allows for metadata to be attached to points. Used for goko. Currently this accelerates distance calculations with a set of packed_simd accelerated norms and a rayon threadpool while abstracting the access of the datapoints across multiple data files. It's structured in such a way that adding formats should be easy.

Planned Features

Current work

  • Benchmarks.
  • PCA, & Gaussian calculators.

Near Future

  • Cleanup of the metadata feature in pointcloud
  • Sparse accessors and sparse databacking

Future

  • Network interface for distributed datasets.
  • Image file abstraction for applications like imagenet.
  • Asynchronous access for the network and file accessors.

lib.rs:

Point Cloud

Abstracts data access over several files and glues metadata files to vector data files

Dependencies

~7.5MB
~140K SLoC