35 releases (13 breaking)
0.14.0 | Sep 18, 2023 |
---|---|
0.13.1 | Aug 29, 2023 |
0.12.3 | Jul 18, 2023 |
0.6.0 | Mar 30, 2023 |
0.1.0 | Mar 5, 2022 |
#249 in Encoding
21,557 downloads per month
Used in 3 crates
(via datafusion-substrait)
47KB
344 lines
substrait-rs
Rust crate for Substrait: Cross-Language Serialization for Relational Algebra.
Documentation
lib.rs
:
Substrait: Cross-Language Serialization for Relational Algebra
Serialization and deserialization
This crate provides generated types to serialize and deserialize Substrait data.
Protobuf
Protobuf serialization and deserialization are provided via [prost] in the [proto] module.
Example
Serialize and deserialize a plan
use prost::Message;
use substrait::proto::Plan;
let plan = Plan::default();
// Serialize the plan
let encoded = plan.encode_to_vec();
// Deserialize the buffer to a Plan
let decoded = Plan::decode(encoded.as_slice())?;
assert_eq!(plan, decoded);
Serde support
There are two (non-default) features available that provide derived Deserialize and Serialize implementations for the generated types.
Note that these features are mutually exclusive. The main difference between
those implementations are the field name case convention and field value
encoding. The examples below show how the minor_number
field name in
Version matches the Protobuf field name with the serde
feature whereas it expects a lower camel case minorNumber
field name with
the pbjson
feature enabled. Please refer to the Protobuf JSON Mapping
documentation for more details.
serde
This adds #[serde(Deserialize, Serialize)]
to all generated Protobuf
types. In addition, to match Protobuf defaults for missing optional data,
this adds [serde(default)]
to all messages.
Example
Deserialize a plan version using the serde
feature
use substrait::proto::Version;
let version_json = r#"{
"minor_number": 21
}"#;
let version = serde_json::from_str::<Version>(version_json)?;
assert_eq!(
version,
Version {
minor_number: 21,
..Default::default()
}
);
pbjson
This generates serde implementation that match the Protobuf JSON Mapping via pbjson.
Example
Deserialize a plan version using the pbjson
feature
use substrait::proto::Version;
let version_json = r#"{
"minorNumber": 21
}"#;
let version = serde_json::from_str::<Version>(version_json)?;
assert_eq!(
version,
Version {
minor_number: 21,
..Default::default()
}
);
Text
Substrait defines a YAML schema for extensions. Types with serialization and deserialization support for these are provided via typify in the [text] module.
Example
Read a simple extension
use substrait::text::simple_extensions::SimpleExtensions;
let simple_extension_yaml = r#"
%YAML 1.2
---
scalar_functions:
-
name: "add"
description: "Add two values."
impls:
- args:
- name: x
value: i8
- name: y
value: i8
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
return: i8
"#;
let simple_extension = serde_yaml::from_str::<SimpleExtensions>(simple_extension_yaml)?;
assert_eq!(simple_extension.scalar_functions.len(), 1);
assert_eq!(simple_extension.scalar_functions[0].name, "add");
Dependencies
~1.7–6.5MB
~122K SLoC