#extractor #parser #dissect

dissect

Logstash inspired dissect extractor

14 releases

Uses new Rust 2024

new 0.7.4 Apr 27, 2025
0.7.3 Oct 26, 2023
0.7.1 Sep 26, 2023
0.7.0 May 24, 2023
0.1.2 Mar 4, 2020

#8 in #extractor

Download history 158/week @ 2025-01-07 166/week @ 2025-01-14 199/week @ 2025-01-21 36/week @ 2025-01-28 100/week @ 2025-02-04 185/week @ 2025-02-11 135/week @ 2025-02-18 115/week @ 2025-02-25 54/week @ 2025-03-04 111/week @ 2025-03-11 93/week @ 2025-03-18 97/week @ 2025-03-25 65/week @ 2025-04-01 59/week @ 2025-04-08 92/week @ 2025-04-15 203/week @ 2025-04-22

425 downloads per month
Used in 2 crates (via tremor-script)

Apache-2.0

66KB
1.5K SLoC

Rust 1K SLoC // 0.0% comments Shell 235 SLoC // 0.0% comments

Dissect   Latest Version Build Status Quality Checks License Checks Security Checks Code Coverage

dissect parser


Dissect parsing inspired by logstash's dissect plugin.

Parses a string into a map.

Use as a library

The dissect parser was designed so that KV style parsing could be embedded into tremor's scripting language for extract operations.

The parser can also be used standalone. A fairly gnarly example of parsing logs from this libraries tests illustrates better than words can:

  let pattern = r#"%{syslog_timestamp} %{syslog_hostname} %{?syslog_prog}: %{syslog_program_aux}[%{syslog_pid:int}] %{request_unix_time} %{request_timestamp} %{request_elapsed_time} %{server_addr}:%{server_port:int} %{remote_addr}:%{remote_port:int} "%{response_content_type}" %{response_content_length} %{request_status} %{bytes_sent} %{request_length} "%{url_scheme}" "%{http_host}" "%{request_method} %{request_url} %{request_protocol}" "%{http_referer}" "%{http_user_agent}" "%{http_x_forwarded_for}" "%{http_ttrue_client_ip}" "%{remote_user}" "%{is_bot}" "%{admin_user}" "%{http_via}" "%{response_location}" "%{set_cookie}" "%{http_cookie}" "%{moawsl_info}" "%{php_message}" "%{akamai_edgescape}" "%{uid_info}" "%{geoip_country}" "%{geoip_region}" "%{geoip_city}" "%{geoip_postal}" "%{geoip_dma}" "%{server_id}" "%{txid}" "%{hpcnt}" "%{client_accept}" "%{client_accept_charset}" "%{client_accept_encoding}" "%{client_accept_language}" "%{client_accept_datetime}" "%{client_pragma}" "%{client_transfer_encoding}" "%{client_attdeviceid}" "%{client_wap_profile}" %{weblog_end}"#;
  let p = lex("%{name}%{_}%{_(|)}%{age}");
  assert!(lex(pattern).is_ok());
  assert!(p.is_ok());

Dependencies

~2.3–3.5MB
~66K SLoC