#apache-arrow #query-execution #data-access #arrow #query #sql-query #sql

datafusion-data-access

General data access layer currently mainly based on the object store interfaces

3 stable releases

10.0.0 Jul 15, 2022
9.0.0 Jun 13, 2022
8.0.0 May 16, 2022

#16 in #query-execution

Download history 1/week @ 2023-11-27 1/week @ 2023-12-11 5/week @ 2023-12-18 6/week @ 2023-12-25 6/week @ 2024-01-08 1/week @ 2024-02-05 33/week @ 2024-02-19 47/week @ 2024-02-26 13/week @ 2024-03-04 36/week @ 2024-03-11

129 downloads per month
Used in 4 crates (via datafusion-objectstore-s3)

Apache-2.0

22KB
361 lines

DataFusion Data Access Layer

DataFusion is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.

This crate is a submodule of DataFusion that provides an async API for accessing data, either remotely or locally. Currently, it is based on the object store interfaces. In the future, this module may include interfaces for accessing databases, or streaming data.

Dependencies

~6–18MB
~224K SLoC