URL Preview

A high-performance Rust library for generating rich URL previews with specialized support for Twitter/X and GitHub. This library offers efficient caching, concurrent processing, and comprehensive metadata extraction capabilities.

Features

• High Performance: Optimized for speed with concurrent processing and batch operations
• Smart Caching: Built-in DashMap-based caching system for lightning-fast responses
• Platform-Specific Handlers:
  • Twitter/X: Specialized handling with oEmbed support
  • GitHub: Enhanced repository information extraction
• Flexible Configuration:
  • Customizable HTTP clients with different configurations
  • Adjustable concurrent request limits
  • Configurable cache sizes
• Rich Metadata Extraction:
  • Title, description, and images
  • Open Graph and Twitter Card metadata
  • Favicons and site information
• Robust Error Handling:
  • Structured error types
  • Detailed logging with tracing support
  • Rate limiting protection
• Modern Rust Features:
  • Async/await with Tokio
  • Thread-safe components
  • Zero-cost abstractions

Installation

Add this to your Cargo.toml:

[dependencies]
url_preview = "0.1.0"

Quick Start

Here's a simple example to get started:

use url_preview::{PreviewService, Preview, PreviewError};

#[tokio::main]
async fn main() -> Result<(), PreviewError> {
    // Create a preview service with default settings
    // Equivalent to：
    //    PreviewService::with_cache_cap(1000)
    let preview_service = PreviewService::new();

    // Generate a preview
    let preview = preview_service
        .generate_preview("https://www.rust-lang.org")
        .await?;

    println!("Title: {:?}", preview.title);
    println!("Description: {:?}", preview.description);
    println!("Image: {:?}", preview.image_url);

    Ok(())
}

Advanced Usage

Batch Processing

Process multiple URLs efficiently:

use url_preview::{PreviewService, Fetcher};

let service = PreviewService::new();
let urls = vec![
    "https://www.rust-lang.org",
    "https://github.com/rust-lang/rust"
];

// Using batch fetching
let results = service.default_generator.fetcher
    .fetch_batch(urls)
    .await?;

// Or using concurrent processing
let results = futures::future::join_all(
    urls.iter().map(|url|
        service.generate_preview_with_concurrency(url)
    )
).await;

Custom Configuration

Configure the service with specific requirements:

use url_preview::{PreviewService, PreviewServiceConfig, Fetcher, FetcherConfig};
use std::time::Duration;

let config = PreviewServiceConfig::new(1000) // Cache capacity
    .with_default_fetcher(
        Fetcher::new_with_config(FetcherConfig {
            timeout: Duration::from_secs(30),
            user_agent: "custom-agent/1.0".into(),
            ..Default::default()
        })
    );

let service = PreviewService::new_with_config(config);

Specialized Platform Support

Twitter/X Integration

let preview = service
    .generate_preview("https://x.com/username/status/123456789")
    .await?;

// Automatic oEmbed support
println!("Tweet content: {:?}", preview.description);

GitHub Repository Information

let preview = service
    .generate_preview("https://github.com/owner/repo")
    .await?;

// Get detailed repository information
let details = service
    .get_github_detailed_info("https://github.com/owner/repo")
    .await?;

println!("Stars: {}", details.stars_count);
println!("Forks: {}", details.forks_count);

Logging Configuration

Configure comprehensive logging:

use url_preview::{setup_logging, LogConfig};
use std::path::PathBuf;

let log_config = LogConfig {
    log_dir: PathBuf::from("logs"),
    log_level: "info".into(),
    console_output: true,
    file_output: true,
};

setup_logging(log_config);

Performance Optimization

Concurrent Request Limiting

Control the number of concurrent requests:

let config = PreviewServiceConfig {
    max_concurrent_requests: 10,
    cache_capacity: 1000,
    ..Default::default()
};

let service = PreviewService::new_with_config(config);

Caching Strategy

The library uses DashMap for thread-safe, high-performance caching:

let cache = Cache::new(1000); // Configure cache size

// Cache operations are automatic in PreviewService
// Manual cache operations if needed:
cache.set("key".to_string(), preview).await;
let cached_preview = cache.get("key").await;

Error Handling

The library provides comprehensive error handling:

match service.generate_preview(url).await {
    Ok(preview) => {
        println!("Successfully generated preview");
    }
    Err(PreviewError::RateLimitError(msg)) => {
        println!("Rate limit exceeded: {}", msg);
    }
    Err(PreviewError::FetchError(msg)) => {
        println!("Failed to fetch content: {}", msg);
    }
    Err(e) => {
        println!("Other error: {}", e);
    }
}

Contributing

Contributions are welcome! Please feel free to submit issues and pull requests.

Development Setup

1. Clone the repository
2. Install dependencies: cargo build
3. Run tests: cargo test
4. Run benchmarks: cargo bench

License

This project is licensed under the MIT License - see the LICENSE file for details.

Testing Websites for Rich Results

Google provides the Rich Results Analysis Tool to help you validate your website's tags.

Use this tool to make sure the website follows these conventions.