substrait-rs

Rust crate for Substrait: Cross-Language Serialization for Relational Algebra.

Documentation

• Docs (release)
• Docs (main)

lib.rs:

Substrait: Cross-Language Serialization for Relational Algebra

Serialization and deserialization

This crate provides generated types to serialize and deserialize Substrait data.

Protobuf

Protobuf serialization and deserialization are provided via [prost] in the [proto] module.

Example

Serialize and deserialize a plan

use prost::Message;
use substrait::proto::Plan;

let plan = Plan::default();

// Serialize the plan
let encoded = plan.encode_to_vec();

// Deserialize the buffer to a Plan
let decoded = Plan::decode(encoded.as_slice())?;

assert_eq!(plan, decoded);

Serde support

There are two (non-default) features available that provide derived Deserialize and Serialize implementations for the generated types.

Note that these features are mutually exclusive. The main difference between those implementations are the field name case convention and field value encoding. The examples below show how the minor_number field name in Version matches the Protobuf field name with the serde feature whereas it expects a lower camel case minorNumber field name with the pbjson feature enabled. Please refer to the Protobuf JSON Mapping documentation for more details.

serde

This adds #[serde(Deserialize, Serialize)] to all generated Protobuf types. In addition, to match Protobuf defaults for missing optional data, this adds [serde(default)] to all messages.

Example

Deserialize a plan version using the serde feature

use substrait::proto::Version;

let version_json = r#"{
  "minor_number": 21
}"#;

let version = serde_json::from_str::<Version>(version_json)?;
assert_eq!(
  version,
  Version {
    minor_number: 21,
    ..Default::default()
  }
);

pbjson

This generates serde implementation that match the Protobuf JSON Mapping via pbjson.

Example

Deserialize a plan version using the pbjson feature

use substrait::proto::Version;

let version_json = r#"{
  "minorNumber": 21
}"#;

let version = serde_json::from_str::<Version>(version_json)?;
assert_eq!(
  version,
  Version {
    minor_number: 21,
    ..Default::default()
  }
);

Text

Substrait defines a YAML schema for extensions. Types with serialization and deserialization support for these are provided via typify in the [text] module.

Example

Read a simple extension

use substrait::text::simple_extensions::SimpleExtensions;

let simple_extension_yaml = r#"
%YAML 1.2
---
scalar_functions:
  -
    name: "add"
    description: "Add two values."
    impls:
      - args:
         - name: x
           value: i8
         - name: y
           value: i8
        options:
          overflow:
            values: [ SILENT, SATURATE, ERROR ]
        return: i8
"#;

let simple_extension = serde_yaml::from_str::<SimpleExtensions>(simple_extension_yaml)?;

assert_eq!(simple_extension.scalar_functions.len(), 1);
assert_eq!(simple_extension.scalar_functions[0].name, "add");