#twitter #rate-limiting #stream #async-stream #helper #user #mode

egg-mode-extras

Rate-limited streams and other helpers for egg-mode

12 releases

0.4.0 Jun 25, 2023
0.3.4 Jan 14, 2023
0.3.3 Nov 11, 2022
0.3.1 Aug 25, 2022
0.1.1 Feb 12, 2022

#824 in Asynchronous

21 downloads per month

MPL-2.0 license

48KB
1K SLoC

egg-mode-extras

Rust build status Coverage status

This project includes some tools that can be useful for working with egg-mode, a Rust library for accessing the Twitter API.

In particular it includes rate-limit-aware asynchronous streams, which make it easy to request e.g. user profiles for millions of Twitter accounts without worrying about Twitter's rate limits:

use egg_mode_extras::{client::TokenType, Client};
use futures::stream::TryStreamExt;
use std::io::BufRead;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::from_config_file("keys.toml").await?;

    let user_ids = std::io::stdin()
        .lock()
        .lines()
        .filter_map(|line| {
            line.map_or_else(
                |error| Some(Err(error)),
                |line| line.parse::<u64>().ok().map(Ok),
            )
        })
        .collect::<Result<Vec<_>, _>>()?;

    client.lookup_users_json(user_ids, TokenType::App).try_for_each(|user| async move {
        println!("{}", user);

        Ok(())
    }).await?;

    Ok(())
}

The code is a mess and is largely untested, although most of it has been abstracted out of ✨cancel-culture✨, where it has been used fairly extensively.

License

This project is licensed under the Mozilla Public License, version 2.0. See the LICENSE file for details.

Please note that we are only using the MPL in order to support use from ✨cancel-culture✨, which is currently published under the MPL. Future versions of both ✨cancel-culture✨ and this project are likely to be published under the Anti-Capitalist Software License.

Dependencies

~13–27MB
~422K SLoC