9 releases

0.0.11 Nov 29, 2022
0.0.10 Nov 21, 2022
0.0.9 Oct 13, 2022
0.0.8 Aug 16, 2022
0.0.1 Mar 30, 2022

#2 in #substrait

Download history 3/week @ 2024-02-23 1/week @ 2024-03-01 1/week @ 2024-03-22 125/week @ 2024-03-29 2/week @ 2024-04-05

128 downloads per month

Apache-2.0

1MB
22K SLoC

Substrait query plan validator

This crate implements a validator for Substrait query plans.

[dependencies]
substrait-validator = "0.0.11"

YAML file resolution

One of the complexities of validating Substrait plans is resolving the YAML extension files. By default, the crate only supports file://... URLs, but often, the YAML files will be stored remotely. To make handling this easier, you can enable curl as an optional dependency:

[dependencies]
substrait-validator = { version = "0.0.11", features = ["curl"] }

This adds the substrait_validator::Config::add_curl_yaml_uri_resolver() method, which will use libcurl to resolve the files, thus supporting all the common protocols (http, https, ftp, etc.). The downside is that the curl crate depends on system libraries.


lib.rs:

Crate for validating Substrait.

Usage

The usage pattern is roughly as follows.

  1. Build a Config structure to configure the validator. You can also just use std::default::Default if you don't need to configure anything, but you might want to at least call [Config::add_curl_uri_resolver()] (if you're using the curl feature).
  2. Parse the incoming substrait.Plan message using [parse()]. This creates a [ParseResult], containing a tree structure corresponding to the query plan that also contains diagnostics and other annotations added by the validator.
  3. You can traverse the tree yourself using ParseResult::root, or you can use one of the methods associated with [ParseResult] to obtain the validation results you need.

Note that only the binary protobuf serialization format is supported at the input; the JSON format is not supported. This is a limitation of prost, the crate that was used for protobuf deserialization. If you're looking for a library (or CLI) that supports more human-friendly input, check out the Python bindings.

Dependencies

~15–29MB
~475K SLoC