Redpanda Data Transforms Rust SDK

This crate contains the SDK for Redpanda's inline Data Transforms, based on WebAssembly.

Data transforms let you run common data streaming tasks, like filtering, scrubbing, and transcoding, within Redpanda. For example, you may have consumers that require you to redact credit card numbers or convert JSON to Avro.

Data transforms use a WebAssembly (Wasm) engine inside a Redpanda broker. A Wasm function acts on a single record in an input topic. You can develop and manage data transforms with rpk transform commands.

See examples/ for sample usages.

lib.rs:

Redpanda Data Transforms Rust library.

Data transforms let you run common data streaming tasks, like filtering, scrubbing, and transcoding, within Redpanda. For example, you may have consumers that require you to redact credit card numbers or convert JSON to Avro.

Data transforms use a WebAssembly (Wasm) engine inside a Redpanda broker. Thus, transforms must compile to WebAssembly via --target=wasm32-wasi. A Wasm function acts on a single record in an input topic. You can develop and manage data transforms with rpk transform commands.

This crate provides a framework for writing transforms.

on_record_written: Transforms individual records after they have been written to an input topic. Written records are output to the destination topic.