#arrow #query #sql

datafusion-data-access

General data access layer currently mainly based on the object store interfaces

3 stable releases

Uses new Rust 2021

10.0.0 Jul 15, 2022
9.0.0 Jun 13, 2022
8.0.0 May 16, 2022

#6 in #big-data

Download history 376/week @ 2022-05-15 435/week @ 2022-05-22 461/week @ 2022-05-29 406/week @ 2022-06-05 556/week @ 2022-06-12 346/week @ 2022-06-19 355/week @ 2022-06-26 480/week @ 2022-07-03 401/week @ 2022-07-10 397/week @ 2022-07-17 465/week @ 2022-07-24 598/week @ 2022-07-31

1,898 downloads per month
Used in 12 crates (via datafusion-objectstore-s3)

Apache-2.0

21KB
411 lines

DataFusion Data Access Layer

DataFusion is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.

This crate is a submodule of DataFusion that provides an async API for accessing data, either remotely or locally. Currently, it is based on the object store interfaces. In the future, this module may include interfaces for accessing databases, or streaming data.

Dependencies

~4–10MB
~165K SLoC