EventHeader for Rust

eventheader is a utility crate implementing tracing infrastructure for writing Linux Tracepoints via user_events. (Generally not to be used directly - use eventheader_dynamic.)

lib.rs:

EventHeader-encoded Linux Tracepoints

The eventheader crate provides a simple and efficient way to log EventHeader-encoded Tracepoints via the Linux user_events system.

This crate uses macros to generate event metadata at compile-time, improving runtime performance and minimizing dependencies. To enable compile-time metadata generation, the event schema must be specified at compile-time. For example, event name and field names must be string literals, not variables.

In rare cases, you might not know what events you want to log until runtime. For example, you might be implementing a middle-layer library providing event support to a dynamic top-layer or a scripting language like JavaScript or Python. In these cases, you might use the eventheader_dynamic crate instead of this crate.

Configuration

• Linux kernel 6.4 or later, with user_events support enabled (CONFIG_USER_EVENTS=y).
• Must have either tracefs or debugfs mounted. For example, you might add the following line to your /etc/fstab file: tracefs /sys/kernel/tracing tracefs defaults 0 0
• The user that will generate events must have x access to the tracing directory, e.g. chmod a+x /sys/kernel/tracing
• The user that will generate events must have rw access to the tracing/user_events_data file, e.g. chmod a+rw /sys/kernel/tracing/user_events_data
• Collect traces using a tool like perf.
• Decode traces using a tool like decode-perf.

Overview

• Use [define_provider!] to create a static symbol for your [Provider], e.g.

  define_provider!(MY_PROVIDER, "MyCompany_MyComponent");

• During component initialization, register the provider, e.g.

  unsafe { MY_PROVIDER.register(); }

  • Safety: [Provider::register] is unsafe because all providers registered by an unloadable shared object must be properly unregistered before the shared object unloads. Since the provider is static, Rust will not automatically drop it. If you are writing an application (not a shared object), this is not a safety issue because the process exits before the application binary unloads.

• As needed, use [write_event!] to send events to user_events.

• During component shutdown, unregister the provider, e.g.

  MY_PROVIDER.unregister();

Example

use eventheader as eh;

// Define a static variable for the "MyCompany_MyComponent" provider.
eh::define_provider!(
    MY_PROVIDER,              // The static symbol to use for this provider.
    "MyCompany_MyComponent"); // The provider's name (string literal).

// Register the provider at component initialization. If you don't register (or if
// register fails) then MY_PROVIDER.enabled() will always return false, the
// write_event! macro will be a no-op, and MY_PROVIDER.unregister() will be a no-op.
// Safety: If this is a shared object, you MUST call MY_PROVIDER.unregister() before unload.
unsafe { MY_PROVIDER.register(); }

// As necessary, call write_event! to send events to user_events.
let field1_value = "String Value";
let field2_value = 42u32;
eh::write_event!(
    MY_PROVIDER,                    // The provider to use for the event.
    "MyEventName",                  // The event's name (string literal).
    level(Warning),                 // Event's severity level.
    keyword(0x23),                  // Event category bits.
    str8("Field1", field1_value),   // Add a string field to the event.
    u32("Field2", &field2_value),   // Add an integer field to the event.
);

// Before module unload, unregister the provider.
MY_PROVIDER.unregister();

Notes

Field value expressions will only be evaluated if the event is enabled, i.e. only if one or more perf logging sessions have enabled the tracepoint that corresponds to the provider + level + keyword combination of the event.

Perf events are limited in size (event size = headers + metadata + data). The system will ignore any event that is larger than 64KB.

Collect the events using a tool like perf. Decode the events using a tool like decode-perf.

Note that you cannot enable a tracepoint until the tracepoint has been registered. Most programs will register all of their tracepoints when they start running, so you can run the program once and then create the session to collect the events. As an alternative, you can pre-register eventheader-based tracepoints using a tool like eventheader-register.

For example, to collect and decode events with level=5 and keyword=1 from a provider that was defined as define_provider!(MY_PROVIDER, "MyCompany_MyComponent"):

perf record -e user_events:MyCompany_MyComponent_L5K1
<run your program>
<Ctrl-C to stop the perf tool>
decode-perf perf.data > perf.json

EventHeader Technical Details

EventHeader is a tracing convention layered on top of Linux Tracepoints.

To reduce the number of unique Tracepoint names tracked by the kernel, we use a small number of Tracepoints to manage a larger number of events. All events with the same attributes (provider name, severity level, category keyword, etc.) will share one Tracepoint.

• This means we cannot enable/disable events individually. Instead, all events with similar attributes will be enabled/disabled as a group.
• This means we cannot rely on the kernel's Tracepoint metadata for event identity or event field names/types. Instead, all events contain a common header that provides event identity and optional field information. The kernel's Tracepoint metadata is used only for the Tracepoint's name and to determine whether the event follows the EventHeader conventions.

We define a naming scheme to be used for the shared Tracepoint:

ProviderName + '_' + 'L' + eventLevel + 'K' + eventKeyword + [Options]

We define a common event layout to be used by all EventHeader events. The event has a header, optional header extensions, and then the event data:

Event = eventheader + [HeaderExtensions] + Data

We define a format to be used for header extensions:

HeaderExtension = eventheader_extension + ExtensionData

We define a header extension to be used for activity IDs.

We define a header extension to be used for event metadata (event name, field names, field types).

For use in the event metadata extension, we define a field type system that supports scalar, string, binary, array, and struct.

Note that we assume that the Tracepoint name corresponding to the event is available during event decoding. The event decoder obtains the provider name and keyword for an event by parsing the event's Tracepoint name.

Provider Names

A provider is a component that generates events. Each event from a provider is associated with a Provider Name that uniquely identifies the provider.

The provider name should be short, yet descriptive enough to minimize the chance of collision and to help developers track down the component generating the events. Hierarchical namespaces may be useful for provider names, e.g. "MyCompany_MyOrg_MyComponent".

Restrictions:

• ProviderName may not contain ' ' or ':' characters.
• strlen(ProviderName + '_' + Attributes) must be less than 256 characters.
• Some event APIs (e.g. tracefs) might impose additional restrictions on tracepoint names. For best compatibility, use only ASCII identifier characters [A-Za-z0-9_] in provider names.

Event attribute semantics should be consistent within a given provider. While some event attributes have generally-accepted semantics (e.g. level value 3 is defined below as "warning"), the precise semantics of the attribute values are defined at the scope of a provider (e.g. different providers will use different criteria for what constitutes a warning). In addition, some attributes (tag, keyword) are completely provider-defined. All events with a particular provider name should use consistent semantics for all attributes (e.g. keyword bit 0x1 should have a consistent meaning for all events from a particular provider but will mean something different for other providers).

Tracepoint Names

A Tracepoint is registered with the kernel for each unique combination of ProviderName + Attributes. This allows a larger number of distinct events to be controlled by a smaller number of kernel Tracepoints while still allowing events to be enabled/disabled at a reasonable granularity.

The Tracepoint name for an EventHeader event is defined as:

ProviderName + '_' + 'L' + eventLevel + 'K' + eventKeyword + [Options] or printf("%s_L%xK%lx%s", providerName, eventLevel, eventKeyword, options), e.g. "MyProvider_L3K2a" or "OtherProvider_L5K1fGperf".

Event level is a uint8 value 1..255 indicating event severity, formatted as lowercase hexadecimal, e.g. printf("L%x", eventLevel). The defined level values are: 1 = critical error, 2 = error, 3 = warning, 4 = information, 5 = verbose.

Event keyword is a uint64 bitmask indicating event category membership, formatted as lowercase hexadecimal, e.g. printf("K%lx", eventKeyword). Each bit in the keyword corresponds to a provider-defined category, e.g. a provider might define 0x2 = networking and 0x4 = I/O so that keyword value of 0x2|0x4 = 0x6 would indicate that an event is in both the networking and I/O categories.

Options (optional attributes) can be specified after the keyword attribute. Each option consists of an uppercase ASCII letter (option type) followed by 0 or more ASCII digits or lowercase ASCII letters (option value). To support consistent event names, the options must be sorted in alphabetical order, e.g. "Aoption" should come before "Boption".

The currently defined options are:

• 'G' = provider Group name. Defines a group of providers. This can be used by event analysis tools to find all providers that generate a certain kind of information.

Restrictions:

• ProviderName may not contain ' ' or ':' characters.
• Tracepoint name must be less than 256 characters in length.
• Some event APIs (e.g. tracefs) might impose additional restrictions on tracepoint names. For best compatibility, use only ASCII identifier characters [A-Za-z0-9_] in provider names.

Header

Because multiple events may share a single Tracepoint, each event must contain information needed to distinguish it from other events. To enable this, each event starts with an EventHeader structure which contains information about the event:

• flags: Bits indicating pointer size (32 or 64 bits), byte order (big-endian or little), and whether any header extensions are present.
• opcode: Indicates special event semantics e.g. "normal event", "activity start event", "activity end event".
• tag: Provider-defined 16-bit value. Can be used for anything.
• id: 16-bit stable event identifier, or 0 if no identifier is assigned.
• version: 8-bit event version, incremented for e.g. field type changes.
• level: 8-bit event severity level, 1 = critical .. 5 = verbose. (level value in event header must match the level in the Tracepoint name.)

If the Extension flag is not set, the header is immediately followed by the event payload.

If the Extension flag is set, the header is immediately followed by one or more EventHeaderExtension blocks. Each header extension has a 16-bit size, a 15-bit type code, and a 1-bit flag indicating whether another header extension block follows the current extension. The final header extension block is immediately followed by the event payload.

The following header extensions are defined:

• Activity ID: Contains a 128-bit ID that can be used to correlate events. May also contain the 128-bit ID of the parent activity (typically used only for the first event of an activity).
• Metadata: Contains the event's metadata - event name, event attributes, field names, field attributes, and field types. Both simple (e.g. Int32, HexInt16, Float64, Char32, Uuid) and complex (e.g. NulTerminatedString8, CountedString16, Binary, Struct, Array) types are supported.