2 unstable releases
0.2.0 | May 23, 2023 |
---|---|
0.1.0 | May 9, 2023 |
#342 in Debugging
75 downloads per month
Used in eventheader_dynamic
150KB
1.5K
SLoC
EventHeader for Rust
eventheader
is a utility crate implementing tracing infrastructure for
writing Linux Tracepoints via user_events
. (Generally not to be used
directly - use eventheader_dynamic
.)
lib.rs
:
EventHeader-encoded Linux Tracepoints
The eventheader
crate provides a simple and efficient way to log
EventHeader-encoded
Tracepoints
via the Linux user_events
system.
This crate uses macros to generate event metadata at compile-time, improving runtime performance and minimizing dependencies. To enable compile-time metadata generation, the event schema must be specified at compile-time. For example, event name and field names must be string literals, not variables.
In rare cases, you might not know what events you want to log until runtime. For
example, you might be implementing a middle-layer library providing event support to a
dynamic top-layer or a scripting language like JavaScript or Python. In these cases,
you might use the eventheader_dynamic
crate instead of this crate.
Configuration
- Linux kernel 6.4 or later, with
user_events
support enabled (CONFIG_USER_EVENTS=y
). - Must have either
tracefs
ordebugfs
mounted. For example, you might add the following line to your/etc/fstab
file:tracefs /sys/kernel/tracing tracefs defaults 0 0
- The user that will generate events must have
x
access to thetracing
directory, e.g.chmod a+x /sys/kernel/tracing
- The user that will generate events must have
rw
access to thetracing/user_events_data
file, e.g.chmod a+rw /sys/kernel/tracing/user_events_data
- Collect traces using a tool like
perf
. - Decode traces using a tool like
decode-perf
.
Overview
-
Use [
define_provider!
] to create a static symbol for your [Provider
], e.g.define_provider!(MY_PROVIDER, "MyCompany_MyComponent");
-
During component initialization, register the provider, e.g.
unsafe { MY_PROVIDER.register(); }
- Safety: [
Provider::register
] is unsafe because all providers registered by an unloadable shared object must be properly unregistered before the shared object unloads. Since the provider is static, Rust will not automatically drop it. If you are writing an application (not a shared object), this is not a safety issue because the process exits before the application binary unloads.
- Safety: [
-
As needed, use [
write_event!
] to send events to user_events. -
During component shutdown, unregister the provider, e.g.
MY_PROVIDER.unregister();
Example
use eventheader as eh;
// Define a static variable for the "MyCompany_MyComponent" provider.
eh::define_provider!(
MY_PROVIDER, // The static symbol to use for this provider.
"MyCompany_MyComponent"); // The provider's name (string literal).
// Register the provider at component initialization. If you don't register (or if
// register fails) then MY_PROVIDER.enabled() will always return false, the
// write_event! macro will be a no-op, and MY_PROVIDER.unregister() will be a no-op.
// Safety: If this is a shared object, you MUST call MY_PROVIDER.unregister() before unload.
unsafe { MY_PROVIDER.register(); }
// As necessary, call write_event! to send events to user_events.
let field1_value = "String Value";
let field2_value = 42u32;
eh::write_event!(
MY_PROVIDER, // The provider to use for the event.
"MyEventName", // The event's name (string literal).
level(Warning), // Event's severity level.
keyword(0x23), // Event category bits.
str8("Field1", field1_value), // Add a string field to the event.
u32("Field2", &field2_value), // Add an integer field to the event.
);
// Before module unload, unregister the provider.
MY_PROVIDER.unregister();
Notes
Field value expressions will only be evaluated if the event is enabled, i.e. only if one or more perf logging sessions have enabled the tracepoint that corresponds to the provider + level + keyword combination of the event.
Perf events are limited in size (event size = headers + metadata + data). The system will ignore any event that is larger than 64KB.
Collect the events using a tool like perf
.
Decode the events using a tool like
decode-perf
.
Note that you cannot enable a tracepoint until the tracepoint has been registered.
Most programs will register all of their tracepoints when they start running, so
you can run the program once and then create the session to collect the events.
As an alternative, you can pre-register eventheader-based tracepoints using a tool
like eventheader-register
.
For example, to collect and decode events with level=5 and keyword=1 from a provider
that was defined as define_provider!(MY_PROVIDER, "MyCompany_MyComponent")
:
perf record -e user_events:MyCompany_MyComponent_L5K1
<run your program>
<Ctrl-C to stop the perf tool>
decode-perf perf.data > perf.json
EventHeader Technical Details
EventHeader
is a tracing convention layered on top of Linux Tracepoints.
To reduce the number of unique Tracepoint names tracked by the kernel, we use a small number of Tracepoints to manage a larger number of events. All events with the same attributes (provider name, severity level, category keyword, etc.) will share one Tracepoint.
- This means we cannot enable/disable events individually. Instead, all events with similar attributes will be enabled/disabled as a group.
- This means we cannot rely on the kernel's Tracepoint metadata for event identity or event field names/types. Instead, all events contain a common header that provides event identity and optional field information. The kernel's Tracepoint metadata is used only for the Tracepoint's name and to determine whether the event follows the EventHeader conventions.
We define a naming scheme to be used for the shared Tracepoint:
ProviderName + '_' + 'L' + eventLevel + 'K' + eventKeyword + [Options]
We define a common event layout to be used by all EventHeader events. The event has a header, optional header extensions, and then the event data:
Event = eventheader + [HeaderExtensions] + Data
We define a format to be used for header extensions:
HeaderExtension = eventheader_extension + ExtensionData
We define a header extension to be used for activity IDs.
We define a header extension to be used for event metadata (event name, field names, field types).
For use in the event metadata extension, we define a field type system that supports scalar, string, binary, array, and struct.
Note that we assume that the Tracepoint name corresponding to the event is available during event decoding. The event decoder obtains the provider name and keyword for an event by parsing the event's Tracepoint name.
Provider Names
A provider is a component that generates events. Each event from a provider is associated with a Provider Name that uniquely identifies the provider.
The provider name should be short, yet descriptive enough to minimize the
chance of collision and to help developers track down the component generating
the events. Hierarchical namespaces may be useful for provider names, e.g.
"MyCompany_MyOrg_MyComponent"
.
Restrictions:
- ProviderName may not contain
' '
or':'
characters. strlen(ProviderName + '_' + Attributes)
must be less than 256 characters.- Some event APIs (e.g. tracefs) might impose additional restrictions on
tracepoint names. For best compatibility, use only ASCII identifier
characters
[A-Za-z0-9_]
in provider names.
Event attribute semantics should be consistent within a given provider. While
some event attributes have generally-accepted semantics (e.g. level value 3
is defined below as "warning"), the precise semantics of the attribute values
are defined at the scope of a provider (e.g. different providers will use
different criteria for what constitutes a warning). In addition, some
attributes (tag, keyword) are completely provider-defined. All events with a
particular provider name should use consistent semantics for all attributes
(e.g. keyword bit 0x1
should have a consistent meaning for all events from a
particular provider but will mean something different for other providers).
Tracepoint Names
A Tracepoint is registered with the kernel for each unique combination of
ProviderName + Attributes
. This allows a larger number of distinct events to
be controlled by a smaller number of kernel Tracepoints while still allowing
events to be enabled/disabled at a reasonable granularity.
The Tracepoint name for an EventHeader event is defined as:
ProviderName + '_' + 'L' + eventLevel + 'K' + eventKeyword + [Options]
or printf("%s_L%xK%lx%s", providerName, eventLevel, eventKeyword, options)
,
e.g. "MyProvider_L3K2a"
or "OtherProvider_L5K1fGperf"
.
Event level is a uint8 value 1..255 indicating event severity, formatted as
lowercase hexadecimal, e.g. printf("L%x", eventLevel)
. The defined level values
are: 1 = critical error, 2 = error, 3 = warning, 4 = information, 5 = verbose.
Event keyword is a uint64 bitmask indicating event category membership,
formatted as lowercase hexadecimal, e.g. printf("K%lx", eventKeyword)
. Each
bit in the keyword corresponds to a provider-defined category, e.g. a provider
might define 0x2 = networking and 0x4 = I/O so that keyword value of 0x2|0x4
=
0x6
would indicate that an event is in both the networking and I/O categories.
Options (optional attributes) can be specified after the keyword attribute.
Each option consists of an uppercase ASCII letter (option type) followed by 0
or more ASCII digits or lowercase ASCII letters (option value). To support
consistent event names, the options must be sorted in alphabetical order, e.g.
"Aoption"
should come before "Boption"
.
The currently defined options are:
'G'
= provider Group name. Defines a group of providers. This can be used by event analysis tools to find all providers that generate a certain kind of information.
Restrictions:
- ProviderName may not contain
' '
or':'
characters. - Tracepoint name must be less than 256 characters in length.
- Some event APIs (e.g. tracefs) might impose additional restrictions on
tracepoint names. For best compatibility, use only ASCII identifier
characters
[A-Za-z0-9_]
in provider names.
Header
Because multiple events may share a single Tracepoint, each event must contain
information needed to distinguish it from other events. To enable this, each
event starts with an EventHeader
structure which contains information about
the event:
flags
: Bits indicating pointer size (32 or 64 bits), byte order (big-endian or little), and whether any header extensions are present.opcode
: Indicates special event semantics e.g. "normal event", "activity start event", "activity end event".tag
: Provider-defined 16-bit value. Can be used for anything.id
: 16-bit stable event identifier, or 0 if no identifier is assigned.version
: 8-bit event version, incremented for e.g. field type changes.level
: 8-bit event severity level, 1 = critical .. 5 = verbose. (level value in event header must match the level in the Tracepoint name.)
If the Extension
flag is not set, the header is immediately followed by the
event payload.
If the Extension
flag is set, the header is immediately followed by one or more
EventHeaderExtension
blocks. Each header extension has a 16-bit size, a 15-bit type code,
and a 1-bit flag indicating whether another header extension block follows the
current extension. The final header extension block is immediately followed by the
event payload.
The following header extensions are defined:
- Activity ID: Contains a 128-bit ID that can be used to correlate events. May also contain the 128-bit ID of the parent activity (typically used only for the first event of an activity).
- Metadata: Contains the event's metadata - event name, event attributes, field names, field attributes, and field types. Both simple (e.g. Int32, HexInt16, Float64, Char32, Uuid) and complex (e.g. NulTerminatedString8, CountedString16, Binary, Struct, Array) types are supported.