• tracing-prof
  • Profiling
    • Wall Clock
    • CPU Time
    • Memory
    • Allocations
    • Heap
  • Exporting
    • pprof
    • Pyroscope
  • Performance Overhead

tracing-prof

A library for continuous profiling of Rust applications built around the tracing library.

The library is experimental, untested, unoptimized, may cause crashes, nasal demons or world war three, and is not recommended for production use.

Platform support is limited to Linux for now, but in theory most platform could be supported where std compiles.

Profiling

The library works by installing a layer via tracing-subscriber that can track various metrics for every span.

Metrics are collected when a span is exited or closed depending on the type of the profile.

Wall Clock

The wall clock time in each span is collected the same way as the tracing-subscriber::fmt layer does.

The following is collected:

• wall_idle: The time a span spent in a created but not entered state.
• wall_busy: The time a span spent in an entered state.
• wall: The total time a span spent in any states.

CPU Time

The CPU time is collected using getrusage.

The following is collected:

• cpu_user: The user CPU time a span spent in an entered state.
• cpu_sys: The system CPU time a span spent in an entered state.
• cpu: The total CPU time a span spent in an entered state.

Memory

Allocations

Allocation tracking can be done by registering a global allocator that tags each allocation borrowing ideas from the tracking_allocator crate.

The following is collected:

• alloc_objects: the number of allocations
• alloc_bytes: the number of bytes allocated

Calls to realloc are counted as a deallocation and an allocation.

Heap

If allocation tracking is enabled, memory for each span are retained until all memory for the span is freed. This allows for periodic snapshots of the heap to be taken for each span.

The following is collected:

• inuse_objects: the number of "live" allocations that were not freed
• inuse_bytes: the amount of bytes still allocated

Exporting

pprof

Profiles can be exported to local files in pprof format.

Pyroscope

Profiles can be sent to a Pyroscope server.

Performance Overhead

Honestly no idea. No extensive benchmarks have been done yet, YMMV.

The collection of samples is done whenever a span is closed or exited, as opposed to other profilers that do sample the stack at a fixed interval (e.g. SIGPROF). This means that the overhead is proportional to the number of spans that are being tracked.

If allocation tracking is enabled, 8 additional bytes of memory is used for the tag for every allocation, even if the allocation itself is excluded from tracking. This may inflate the memory usage of your application for very small and frequent allocations. In addition, bookkeeping for various metrics also uses memory.

Every exporter also adds its own overhead, e.g. file I/O or http requests. All the processing and bookkeeping is done in a separate thread and will never block the application.