#logstash #parser #string #map #plugin #extractor


Logstash inspired dissect extractor

13 unstable releases (6 breaking)

0.7.3 Oct 26, 2023
0.7.1 Sep 26, 2023
0.7.0 May 24, 2023
0.5.0 Nov 3, 2022
0.1.2 Mar 4, 2020

#69 in Programming languages

Download history 50/week @ 2024-02-19 186/week @ 2024-02-26 53/week @ 2024-03-04 88/week @ 2024-03-11 8/week @ 2024-03-18 1/week @ 2024-03-25 132/week @ 2024-04-01 230/week @ 2024-04-08 241/week @ 2024-04-15 238/week @ 2024-04-22

841 downloads per month
Used in 2 crates (via tremor-script)


1.5K SLoC

Rust 1K SLoC // 0.0% comments Shell 235 SLoC // 0.0% comments

Dissect   Latest Version Build Status Quality Checks License Checks Security Checks Code Coverage

dissect parser

Dissect parsing inspired by logstash's dissect plugin.

Parses a string into a map.

Use as a library

The dissect parser was designed so that KV style parsing could be embedded into tremor's scripting language for extract operations.

The parser can also be used standalone. A fairly gnarly example of parsing logs from this libraries tests illustrates better than words can:

  let pattern = r#"%{syslog_timestamp} %{syslog_hostname} %{?syslog_prog}: %{syslog_program_aux}[%{syslog_pid:int}] %{request_unix_time} %{request_timestamp} %{request_elapsed_time} %{server_addr}:%{server_port:int} %{remote_addr}:%{remote_port:int} "%{response_content_type}" %{response_content_length} %{request_status} %{bytes_sent} %{request_length} "%{url_scheme}" "%{http_host}" "%{request_method} %{request_url} %{request_protocol}" "%{http_referer}" "%{http_user_agent}" "%{http_x_forwarded_for}" "%{http_ttrue_client_ip}" "%{remote_user}" "%{is_bot}" "%{admin_user}" "%{http_via}" "%{response_location}" "%{set_cookie}" "%{http_cookie}" "%{moawsl_info}" "%{php_message}" "%{akamai_edgescape}" "%{uid_info}" "%{geoip_country}" "%{geoip_region}" "%{geoip_city}" "%{geoip_postal}" "%{geoip_dma}" "%{server_id}" "%{txid}" "%{hpcnt}" "%{client_accept}" "%{client_accept_charset}" "%{client_accept_encoding}" "%{client_accept_language}" "%{client_accept_datetime}" "%{client_pragma}" "%{client_transfer_encoding}" "%{client_attdeviceid}" "%{client_wap_profile}" %{weblog_end}"#;
  let p = lex("%{name}%{_}%{_(|)}%{age}");


~100K SLoC