story-dl

story-dl is a program allowing you to download stories from a multitude of different sites.

Note

This scraper does not support the scraping of characters, pairings, tags, warnings, or anything other than the required information. Or at least not yet.

As of v0.5.0, stroy-dl contains a modified but untested version of Élisabeth Henry's epub-builder.

Supported Sites

• Archive Of Our Own
• FanFiction
• (yah i need to add more...)

Dependencies

• Rust (build)
• C/C++ compiler (build)
• OpenSSL (Unix/Linux and macOS) (runtime)

Command

Usage

story-dl tries to have a simple command line, with just enough to change what you need to but simple enough so that it doesn't require 20 man pages.

Examples

Download story as EPub:

story-dl -u <URL> -o epub

Download all in import file as EPub:

story-dl -f import.json -o epub

import.json:

[
    "<URL>",
    {
        "url": "<URL>"
    }
]

Library

Usage

Add this to your Cargo.toml.

story_dl = { version = "0.3", default-features = false }

This will add story-dl but disabled all crates required by the command version.

Choose what website will be scraped, lets use FanFiction for this example.

// Import required structs.
use story_dl::fanfiction;

#[tokio::main]
async fn main() {
    // Convert story url string into the required Uri.
    let url = "<story url>".parse().expect("Not a valid URL");

    // Start the scraper, this will return the finished story.
    let story = fanfiction::scrape(&url).await.expect("Error scraping story");

    // Scraped information
    println!("Title: {}", story.name);
    println!("Authors: {}", story.authors.join(", "));
    println!("Chapters: {}", story.chapters);
}

Credits

Internally uses a modified version of Gonzih's CrabQuery.

Thanks to Quite Wraith on Archive Of Our Own and FanFiction.net as two of their stories, "Little Cog" and "Fellow Traveler", are used as test data for site scrapping.