35 releases (13 breaking)

0.14.0 Sep 18, 2023
0.13.1 Aug 29, 2023
0.12.3 Jul 18, 2023
0.6.0 Mar 30, 2023
0.1.0 Mar 5, 2022

#249 in Encoding

Download history 6967/week @ 2023-06-07 5233/week @ 2023-06-14 4705/week @ 2023-06-21 5324/week @ 2023-06-28 5113/week @ 2023-07-05 5528/week @ 2023-07-12 5225/week @ 2023-07-19 5550/week @ 2023-07-26 6574/week @ 2023-08-02 6457/week @ 2023-08-09 7767/week @ 2023-08-16 6334/week @ 2023-08-23 4166/week @ 2023-08-30 5443/week @ 2023-09-06 6560/week @ 2023-09-13 4419/week @ 2023-09-20

21,557 downloads per month
Used in 3 crates (via datafusion-substrait)

Apache-2.0

47KB
344 lines

substrait-rs

substrait

crates.io docs.rs

Rust crate for Substrait: Cross-Language Serialization for Relational Algebra.

Documentation


lib.rs:

Substrait: Cross-Language Serialization for Relational Algebra

Serialization and deserialization

This crate provides generated types to serialize and deserialize Substrait data.

Protobuf

Protobuf serialization and deserialization are provided via [prost] in the [proto] module.

Example

Serialize and deserialize a plan

use prost::Message;
use substrait::proto::Plan;

let plan = Plan::default();

// Serialize the plan
let encoded = plan.encode_to_vec();

// Deserialize the buffer to a Plan
let decoded = Plan::decode(encoded.as_slice())?;

assert_eq!(plan, decoded);

Serde support

There are two (non-default) features available that provide derived Deserialize and Serialize implementations for the generated types.

Note that these features are mutually exclusive. The main difference between those implementations are the field name case convention and field value encoding. The examples below show how the minor_number field name in Version matches the Protobuf field name with the serde feature whereas it expects a lower camel case minorNumber field name with the pbjson feature enabled. Please refer to the Protobuf JSON Mapping documentation for more details.

serde

This adds #[serde(Deserialize, Serialize)] to all generated Protobuf types. In addition, to match Protobuf defaults for missing optional data, this adds [serde(default)] to all messages.

Example
Deserialize a plan version using the serde feature
use substrait::proto::Version;

let version_json = r#"{
  "minor_number": 21
}"#;

let version = serde_json::from_str::<Version>(version_json)?;
assert_eq!(
  version,
  Version {
    minor_number: 21,
    ..Default::default()
  }
);

pbjson

This generates serde implementation that match the Protobuf JSON Mapping via pbjson.

Example
Deserialize a plan version using the pbjson feature
use substrait::proto::Version;

let version_json = r#"{
  "minorNumber": 21
}"#;

let version = serde_json::from_str::<Version>(version_json)?;
assert_eq!(
  version,
  Version {
    minor_number: 21,
    ..Default::default()
  }
);

Text

Substrait defines a YAML schema for extensions. Types with serialization and deserialization support for these are provided via typify in the [text] module.

Example

Read a simple extension

use substrait::text::simple_extensions::SimpleExtensions;

let simple_extension_yaml = r#"
%YAML 1.2
---
scalar_functions:
  -
    name: "add"
    description: "Add two values."
    impls:
      - args:
         - name: x
           value: i8
         - name: y
           value: i8
        options:
          overflow:
            values: [ SILENT, SATURATE, ERROR ]
        return: i8
"#;

let simple_extension = serde_yaml::from_str::<SimpleExtensions>(simple_extension_yaml)?;

assert_eq!(simple_extension.scalar_functions.len(), 1);
assert_eq!(simple_extension.scalar_functions[0].name, "add");

Dependencies

~1.7–6.5MB
~122K SLoC