Rust Observer: Observability for High-Performance Systems

Rust Observer is a high-performance, concurrent observability library designed for mission-critical Rust applications. It provides unified logging, tracing, and metrics collection with minimal runtime impact, optimized for systems with strict performance requirements.

Rust Observer is part of the Express Telemetry Project.

Core Principles

1. Concurrency-first design: Built from the ground up for highly concurrent systems.
2. Unified observability: Integrates logging, tracing, and metrics in a cohesive manner.
3. Extensibility: Easily adaptable to custom enterprise environments.

Key Features

• Non-blocking operations: All telemetry operations are non-blocking, preserving application responsiveness.
• Lock-free architecture: Utilizes atomic operations and lock-free data structures to minimize contention.
• Adaptive sampling: Implements intelligent sampling to manage data volume without loss of critical information.
• Buffered, batched exports: Optimizes I/O operations for improved throughput.
• Fine-grained configuration: Offers detailed tuning options via TOML configuration.

Integration

1. Add to Cargo.toml:

   [dependencies]
   rust_observer = "0.2.1"
   

2. Initialize in your application:

   use rust_observer::initialize_observability;
   
   #[tokio::main]
   async fn main() -> Result<(), Box<dyn std::error::Error>> {
       initialize_observability("config/observability.toml")?;
       // Application logic
       Ok(())
   }
   

Usage Examples

Structured Logging

use rust_observer::logging::info;

info!("Transaction processed", attrs: {
    "transaction_id" => tx.id,
    "amount" => tx.amount,
    "currency" => tx.currency,
    "status" => tx.status
});

Distributed Tracing

use rust_observer::tracing::{trace, OptionalTraceContext};

#[trace(Server)]
async fn process_order(ctx: OptionalTraceContext, order: Order) -> Result<(), Error> {
    validate_order(ctx.clone(), &order)?;
    update_inventory(ctx.clone(), &order)?;
    process_payment(ctx.clone(), &order)?;
    Ok(())
}

Interval-based Metrics

use rust_observer::metrics::HttpMetrics;

async fn handle_request(req: Request) -> Response {
    let start = Instant::now();
    let response = process_request(req).await;
    let duration = start.elapsed().as_millis() as u64;

    HttpMetrics::increment_counter().await;
    HttpMetrics::update_method_count(req.method()).await;
    HttpMetrics::update_status_code(response.status()).await;
    HttpMetrics::update_request_duration(req.method(), response.status(), duration).await;

    response
}

Advanced Configuration

Rust Observer supports fine-grained configuration via TOML:

[project]
name = "falcon-9"
app_name = "flight-computer"
service_name = "telemetry-processor"

[logging]
enabled = true
level = "INFO"

[logging.exporter]
type = "stdout"
format = "extended"

[tracing]
enabled = true

[tracing.exporter]
type = "stdout"
format = "extended"

[metrics]
enabled = true
interval_secs = 5

[metrics.exporter]
type = "stdout"
format = "extended"

[metrics.http]
enabled = true

[metrics.grpc]
enabled = true

[metrics.websockets]
enabled = true

[metrics.system]
enabled = true

[metrics.process]
enabled = true

Deployment Considerations

• Supports horizontal scaling with consistent performance characteristics.