#logstash #grok #elastic #log

grok-rs

Rust port of elastic Grok processor

4 releases

0.1.3 Jun 9, 2024
0.1.2 Jun 9, 2024
0.1.1 Jun 7, 2024
0.1.0 Jun 7, 2024

#3 in #logstash

Apache-2.0

39KB
745 lines

Build Status Version codecov

grok_rs

the grok_rs is a rust port of Elastic Grok processor, inspired by grok-go and grok

Usage

[dependencies]
grok-rs = "0.1.3"

Example

Only with default patterns

let grok = Grok::default();
let pattern = grok
    // USERNAME are defined in grok-patterns
    .compile("%{USERNAME}", false)
    .unwrap();
let result = pattern.parse("admin admin@example.com").unwrap();
println!("{:#?}", result);

the output is:

{
    "USERNAME": String(
        "admin",
    ),
}

With user-defined patterns

let mut grok = Grok::default();
grok.add_pattern("NAME", r"[A-z0-9._-]+");
let pattern = grok.compile("%{NAME}", false).unwrap();
let result = pattern.parse("admin").unwrap();
println!("{:#?}", result);

the output is:

{
    "NAME": String(
        "admin",
    ),
}

With named_capture_only is true

let grok = Grok::default();
let pattern = grok
    .compile("%{USERNAME} %{EMAILADDRESS:email}", true)
    .unwrap();
let result = pattern.parse("admin admin@example.com").unwrap();
println!("{:#?}", result);

the output is:

{
    "email": String(
        "admin@example.com",
    ),
}

With type

let mut grok = Grok::default();
grok.add_pattern("NUMBER", r"\d+");

let pattern = grok.compile("%{NUMBER:digit:int}", false).unwrap();
let result = pattern.parse("hello 123").unwrap();
println!("{:#?}", result);

the output is:

{
    "digit": Int(
        123,
    ),
}

Notice

grok_rs is based on regex crate, so lacks several features that are not known how to implement efficiently. This includes, but is not limited to, look-around and backreferences. In exchange, all regex searches in this crate have worst case O(m * n) time complexity, where m is proportional to the size of the regex and n is proportional to the size of the string being searched.

Elastic Grok compliance

This crate declares compatible with elastic grok patterns v8.14.0, which is tagged at 2024-06-05.

Dependencies

~2.2–3MB
~55K SLoC