#datasets

nightly pointcloud

An accessor layer for goko

18 releases

0.5.5 May 4, 2021
0.5.4 Mar 3, 2021
0.5.3 Dec 16, 2020
0.3.9 Sep 17, 2020
0.1.2 Feb 27, 2020

#338 in Science

Download history 16/week @ 2022-03-02 8/week @ 2022-03-09 63/week @ 2022-03-16 78/week @ 2022-03-23 14/week @ 2022-03-30 19/week @ 2022-04-06 10/week @ 2022-04-13 5/week @ 2022-04-20 137/week @ 2022-04-27 115/week @ 2022-05-04 283/week @ 2022-05-11 80/week @ 2022-05-18 83/week @ 2022-05-25 163/week @ 2022-06-01 55/week @ 2022-06-08 5/week @ 2022-06-15

324 downloads per month
Used in 2 crates

Custom license

145KB
3.5K SLoC

Point Cloud

A dataset access layer that allows for metadata to be attached to points. Used for goko. Currently this accelerates distance calculations with a set of packed_simd accelerated norms and a rayon threadpool while abstracting the access of the datapoints across multiple data files. It's structured in such a way that adding formats should be easy.

Planned Features

Current work

  • Benchmarks.
  • PCA, & Gaussian calculators.

Near Future

  • Cleanup of the metadata feature in pointcloud
  • Sparse accessors and sparse databacking

Future

  • Network interface for distributed datasets.
  • Image file abstraction for applications like imagenet.
  • Asynchronous access for the network and file accessors.

lib.rs:

Point Cloud

Abstracts data access over several files and glues metadata files to vector data files

Dependencies

~8MB
~153K SLoC