Lib.rs

› Programming languages
#scripting-language #data-processing #datadog #nlp #compiler #query-language #syntax-tree

vrl

Vector Remap Language

by Jesse Szwedko, Pavlos Rontidis, Nathan Fox, vectordotdev

16 breaking releases

new 0.20.1 Dec 9, 2024
0.20.0 Nov 27, 2024
0.19.0 Oct 1, 2024
0.17.0 Jul 24, 2024
0.5.0 Jun 28, 2023

#57 in Programming languages

Download history 3131/week @ 2024-08-24 2182/week @ 2024-08-31 2329/week @ 2024-09-07 1413/week @ 2024-09-14 1844/week @ 2024-09-21 2164/week @ 2024-09-28 2002/week @ 2024-10-05 2138/week @ 2024-10-12 1919/week @ 2024-10-19 1513/week @ 2024-10-26 1674/week @ 2024-11-02 4691/week @ 2024-11-09 4857/week @ 2024-11-16 4412/week @ 2024-11-23 5002/week @ 2024-11-30 4334/week @ 2024-12-07

19,513 downloads per month
Used in cdviz-collector

MPL-2.0 license

2.5MB
62K SLoC

Vector Remap Language (VRL)

Crates.io docs.rs GitHub Workflow Status

VRL is a scripting language for processing observability data (logs, metrics, traces). Although VRL was originally created for use in Vector, it was designed to be generic and re-usable in many contexts.

Features

VRL is broken up into multiple components, which can be enabled as needed.

Feature Default Description
compiler yes The contains the core functionality of VRL. Compiling and running VRL programs.
parser yes Creates an abstract syntax tree (AST) from VRL source code.
value yes Contains the primary data type used in VRL.
diagnostic yes Logic related to errors and displaying info about them.
path yes Contains the parser, datatypes, and functions related to VRL paths.
stdlib yes All of the VRL functions from the standard library.
core yes Various data structures and utility methods (these may be renamed / moved in the future).
datadog_filter yes Implements the Datadog log search query filter syntax.
datadog_grok yes Implements the Datadog grok parser. (used with parse_grok and parse_groks in the stdlib).
datadog_search yes Implements the Datadog log search syntax.
cli no Contains functionality to create a CLI for VRL.
test_framework no Contains the test framework for testing VRL functions. Useful for testing custom functions.
lua no Makes the Value type compatible with the mlua crate.
arbitrary no Implements Arbitrary (from the quickcheck crate) for the Value type

Webassembly

All of the core features, and most of the standard library functions can be compiled with the wasm32-unknown-unknown target. There are a few stdlib functions that are unsupported. These will still compile, but abort at runtime.

Unsupported functions:

  • parse_grok
  • parse_groks
  • log
  • get_hostname
  • reverse_dns

Dependencies

~5–44MB
~709K SLoC