#wikimedia #toolforge

bin+lib toolforge

Small library for common tasks on Wikimedia Toolforge

17 releases (6 stable)

Uses new Rust 2021

5.5.0 Jan 20, 2023
5.4.0 Dec 27, 2022
5.2.0 Oct 11, 2022
5.1.0 Jan 9, 2022
0.2.0 Sep 15, 2020

#1 in #wikimedia

Download history 31/week @ 2022-10-07 63/week @ 2022-10-14 13/week @ 2022-10-21 6/week @ 2022-10-28 15/week @ 2022-11-04 5/week @ 2022-11-11 16/week @ 2022-11-18 3/week @ 2022-11-25 4/week @ 2022-12-02 28/week @ 2022-12-09 23/week @ 2022-12-16 43/week @ 2022-12-23 19/week @ 2022-12-30 36/week @ 2023-01-06 30/week @ 2023-01-13 43/week @ 2023-01-20

129 downloads per month

GPL-3.0-or-later

36KB
539 lines

toolforge

crates.io docs.rs pipeline status

The toolforge crate provides helper functions for various tasks on Toolforge.

See wikitech for the full documentation.

License

toolforge is (C) 2013, 2017, 2020-2021 Kunal Mehta, released under the GPL v3 or any later version, see COPYING for details.


lib.rs:

The toolforge crate provides helper functions for various tasks on Toolforge.

Constructing a User-Agent

The user_agent! macro generates a tool-specific User-Agent header in compliance with Wikimedia's User-Agent policy. The first and only required parameter is the tool's name. Optional second and third parameters can be overrides to the url and email, respectively.

Example

const USER_AGENT: &str = toolforge::user_agent!("mycooltool");

assert_eq!(
    USER_AGENT,
    "https://mycooltool.toolforge.org/ tools.mycooltool@toolforge.org"
);

WikiReplica connection helpers

The connection_info! macro builds the URL string to connect to Wiki Replicas for use with the mysql_async crate. It should also work with the mysql crate. Specifically, it follows the Toolforge connection handling policy.

The first argument is the database name, with or without a _p suffix. The second (optional) argument is which cluster to connect to, either WEB (default) or ANALYTICS.

Example

use mysql_async::Pool;
# async fn demo() -> toolforge::Result<()> {
let pool = Pool::new(
    toolforge::connection_info!("enwiki", WEB)
        .expect("unable to load db config")
        .to_string()
        .as_str(),
);
let mut conn = pool.get_conn().await?;
# Ok(())
# }

Local development

Copy your tool's replica.my.cnf to your local machine, saving it to ~/replica.my.cnf and add a local='true' flag. For example:

[client]
user='u####'
password='...'
local='true'

Then open a SSH tunnel on port 3306 to the specific wiki replicas database you want to access. The toolforge-tunnel helper described below can simplify this.

toolforge-tunnel helper

The toolforge-tunnel tool simplifies the process of opening SSH tunnels to wiki replicas. Example usage: toolforge-tunnel enwiki.

It can be installed via cargo: cargo install toolforge --features=cli. Pre-built binaries can be downloaded from GitLab.

Unstable features

WikiPool is a wrapper around mysql_async::Pool that given a database name, will connect to the appropriate backend server. It is smart enough to know which databases are on the same server so connections can be optimally reused for better performance. This is behind the unstable-pool feature until the API is finalized.

Contributing

toolforge is a part of the mwbot-rs project. We're always looking for new contributors, please reach out if you're interested!

Dependencies

~0–7MB
~121K SLoC