3 stable releases
Uses new Rust 2021
10.0.0 | Jul 15, 2022 |
---|---|
9.0.0 | Jun 13, 2022 |
8.0.0 | May 16, 2022 |
#310 in Encoding
150 downloads per month
Used in 6 crates
(via ballista-core)
2.5MB
52K
SLoC
DataFusion Proto
DataFusion is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
This crate is a submodule of DataFusion that provides a protocol buffer format for representing query plans and expressions.
Serializing Expressions
Based on examples/expr_serde.rs
use datafusion_common::Result;
use datafusion_expr::{col, lit, Expr};
use datafusion_proto::bytes::Serializeable;
fn main() -> Result<()> {
// Create a new `Expr` a < 32
let expr = col("a").lt(lit(5i32));
// Convert it to an opaque form
let bytes = expr.to_bytes()?;
// Decode bytes from somewhere (over network, etc.)
let decoded_expr = Expr::from_bytes(&bytes)?;
assert_eq!(expr, decoded_expr);
Ok(())
}
Serializing Plans
Based on examples/plan_serde.rs
use datafusion::prelude::*;
use datafusion_common::Result;
use datafusion_proto::bytes::{logical_plan_from_bytes, logical_plan_to_bytes};
#[tokio::main]
async fn main() -> Result<()> {
let ctx = SessionContext::new();
ctx.register_csv("t1", "testdata/test.csv", CsvReadOptions::default())
.await
?;
let plan = ctx.table("t1")?.to_logical_plan()?;
let bytes = logical_plan_to_bytes(&plan)?;
let logical_round_trip = logical_plan_from_bytes(&bytes, &ctx)?;
assert_eq!(format!("{:?}", plan), format!("{:?}", logical_round_trip));
Ok(())
}
Dependencies
~39MB
~797K SLoC