#archive #url #client #machine #wayback #retries #internet

waybackmachine-client

Client to archive URLs in the Internet Archive's Wayback Machine

6 releases

0.4.2 Aug 8, 2024
0.4.1 Jun 17, 2024
0.3.1 May 31, 2024
0.3.0 Mar 27, 2024
0.2.0 Mar 27, 2024

#407 in Compression

33 downloads per month
Used in archive-pdf-urls

Apache-2.0

25KB
487 lines

Wayback Machine Client

This Rust crate provides a client for interacting with the Wayback Machine, allowing users to archive URLs.

Build status Crates.io

Installation

cargo add waybackmachine-client

Usage

The WaybackMachineClient struct provides methods for archiving URLs using the Wayback Machine service. You can use the archive_url method to archive a URL asynchronously.

Example:

use waybackmachine_client::{ClientConfig, Error, WaybackMachineClient};

#[tokio::main]
async fn main() -> Result<(), Error> {
    let wayback_client = WaybackMachineClient::new(ClientConfig::default());
    wayback_client.archive_url("https://www.example.com").await?;
    Ok(())
}

Features

  • Automatic Retry: The client automatically retries failed requests with exponential backoff, configurable via the ClientConfig.max_request_retries setting.
  • Recent Archive Check: The client checks if a URL has been archived within a specified threshold using the ClientConfig.archive_threshold_days setting.
  • Customisable Configuration: You can customise the client's behavior using the ClientConfig struct.
  • Asynchronous: Requests are sent asynchronously using the Tokio runtime

Dependencies

~6–17MB
~239K SLoC