11 releases

0.1.1 Nov 8, 2024
0.1.0 Nov 6, 2024
0.0.11 Nov 29, 2022
0.0.9 Oct 13, 2022
0.0.1 Mar 30, 2022

#3 in #substrait

Download history 15/week @ 2024-09-22 15/week @ 2024-09-29 188/week @ 2024-11-03 55/week @ 2024-11-10 102/week @ 2024-11-17 48657/week @ 2024-11-24 58437/week @ 2024-12-01 153467/week @ 2024-12-08

260,663 downloads per month

Apache-2.0

1MB
23K SLoC

Contains (WOFF font, 155KB) src/export/html/fa-solid-900.woff2

Substrait query plan validator

This crate implements a validator for Substrait query plans.

[dependencies]
substrait-validator = "0.1.1"

YAML file resolution

One of the complexities of validating Substrait plans is resolving the YAML extension files. By default, the crate only supports file://... URLs, but often, the YAML files will be stored remotely. To make handling this easier, you can enable curl as an optional dependency:

[dependencies]
substrait-validator = { version = "0.1.1", features = ["curl"] }

This adds the substrait_validator::Config::add_curl_yaml_uri_resolver() method, which will use libcurl to resolve the files, thus supporting all the common protocols (http, https, ftp, etc.). The downside is that the curl crate depends on system libraries.


lib.rs:

Crate for validating Substrait.

Usage

The usage pattern is roughly as follows.

  1. Build a Config structure to configure the validator. You can also just use std::default::Default if you don't need to configure anything, but you might want to at least call Config::add_curl_uri_resolver() (if you're using the curl feature).
  2. Parse the incoming substrait.Plan message using [parse()] or [validate()]. This creates a [ParseResult], containing a tree structure corresponding to the query plan that also contains diagnostics and other annotations added by the validator.
  3. You can traverse the tree yourself using ParseResult::root, or you can use one of the methods associated with [ParseResult] to obtain the validation results you need.

Note that only the binary protobuf serialization format is supported at the input; the JSON format is not supported. This is a limitation of prost, the crate that was used for protobuf deserialization. If you're looking for a library (or CLI) that supports more human-friendly input, check out the Python bindings.

Dependencies

~15–27MB
~415K SLoC