#tracing #opentelemetry

tracing-opentelemetry-instrumentation-sdk

A set of helpers to build OpenTelemetry instrumentation based on tracing crate

12 releases (5 breaking)

0.17.0 Feb 11, 2024
0.16.0 Dec 30, 2023
0.15.2 Feb 11, 2024
0.15.0 Nov 25, 2023
0.12.0 Jul 2, 2023

#153 in Debugging

Download history 3000/week @ 2023-11-02 3773/week @ 2023-11-09 4067/week @ 2023-11-16 3205/week @ 2023-11-23 3214/week @ 2023-11-30 4090/week @ 2023-12-07 3610/week @ 2023-12-14 1921/week @ 2023-12-21 2781/week @ 2023-12-28 3406/week @ 2024-01-04 4264/week @ 2024-01-11 4553/week @ 2024-01-18 4582/week @ 2024-01-25 4827/week @ 2024-02-01 5675/week @ 2024-02-08 5851/week @ 2024-02-15

21,811 downloads per month
Used in 10 crates (4 directly)

CC0 license

30KB
450 lines

tracing-opentelemetry-instrumentation-sdk

Provide a set of helpers to build OpenTelemetry instrumentation based on tracing crate, and following the OpenTelemetry Trace Semantic Conventions.

PS: Contributions are welcome (bug report, improvements, features, ...)

Instrumentation on the caller side of a call is composed of steps:

  • start a span with all the attributes (some set to Empty)
  • inject into the call (via header) the propagation data (if supported)
  • do the call
  • update attributes of the span with response (status,...)

Instrumentation on the callee side of a call is composed of steps:

  • extract info propagated info (from header) (if supported) an create an OpenTelemetry Context
  • start a span with all the attributes (some set to Empty)
  • attach the context as parent on the span
  • do the processing
  • update attributes of the span with response (status,...)

The crates provide helper (or inspiration) to extract/inject context info, start & update span and retrieve context or trace_id during processing (eg to inject trace_id into log, error message,...).

  let trace_id = tracing_opentelemetry_instrumentation_sdk::find_current_trace_id();
  //json!({ "error" :  "xxxxxx", "trace_id": trace_id})

The helpers could be used as is or into middleware build on it (eg: axum-tracing-opentelemetry, tonic-tracing-opentelemetry are middlewares build on top of the helpers provide for http (feature & crate))

Notes

  • tracing-opentelemetry extends tracing to interoperate with OpenTelemetry. But with some constraints:
    • Creation of the OpenTelemetry's span is done when the tracing span is closed. So do not try to interact with OpenTelemetry Span (or SpanBuilder) from inside the tracing span.
    • The OpenTelemetry parent Context (and trace_id) is created on NEW span or inherited from parent span. The parent context can be overwritten after creation, but until then the trace_id is the one from NEW, So tracing's log could report none or not-yet set trace_id on event NEW and the following until update.
    • To define kind, name,... of OpenTelemetry's span from tracing's span used special record's name: otel.name, otel.kind, ...
    • Record in a tracing's Span should be defined at creation time. So some field are created with value tracing::field::Empty to then being updated.
  • Create trace with target otel::tracing (and level trace), to have a common way to enable / to disable

Instrumentations Tips

Until every crates are instrumented

Use tracing::instrumented (no propagation & no update on response)

// basic handmade span far to be compliant with
//[opentelemetry-specification/.../database.md](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.22.0/specification/trace/semantic_conventions/database.md)
fn make_otel_span(db_operation: &str) -> tracing::Span {
    // NO parsing of statement to extract information, not recommended by Specification and time-consuming
    // warning: providing the statement could leek information
    tracing::trace_span!(
        target: tracing_opentelemetry_instrumentation_sdk::TRACING_TARGET,
        "DB request",
        db.system = "postgresql",
        // db.statement = stmt,
        db.operation = db_operation,
        otel.name = db_operation, // should be <db.operation> <db.name>.<db.sql.table>,
        otel.kind = "CLIENT",
        otel.status_code = tracing::field::Empty,
    )
}


      // Insert or update
        sqlx::query!(
                "INSERT INTO ...",
                id,
                sub_key,
                result,
            )
            .execute(&*self.pool)
            .instrument(make_otel_span("INSERT"))
            .await
            .map_err(...)?;

Dependencies

~6.5MB
~118K SLoC