#database #jobs #tasks #background #postgres

coil

An async and sync background job queue for Postgres

5 unstable releases

new 0.4.0 Jun 15, 2021
0.2.2 Feb 8, 2021
0.2.1 Feb 8, 2021
0.2.0 Sep 3, 2020
0.1.0 Aug 18, 2020

#129 in Database interfaces

Download history 140/week @ 2021-02-25 91/week @ 2021-03-04 44/week @ 2021-03-11 44/week @ 2021-03-18 116/week @ 2021-03-25 4/week @ 2021-04-01 62/week @ 2021-04-08 41/week @ 2021-04-15 19/week @ 2021-04-22 35/week @ 2021-04-29 94/week @ 2021-05-06 28/week @ 2021-05-13 78/week @ 2021-05-20 126/week @ 2021-05-27 98/week @ 2021-06-03 128/week @ 2021-06-10

275 downloads per month

GPL-3.0 license

39KB
804 lines

Coil

An async task queue built with SQLx, Postgres, and Rayon

Coverage Crates.io Crates.io

Coil is heavily inspired by and takes heavily from swirl. In many places of the codebase, code is very similiar.

Supports synchronous and asynchronous jobs. Synchronous jobs will be spawned into a threadpool managed by rayon. Async jobs will be spawned onto an executor. The only requirement is that the executor implements the futures Spawn trait. This way, coil supports Tokio, smol, and async-std.

† This software is alpha, and not intended for production use yet. Use at your own risk.

†† This software is sublicensed as GPLv3. Portions from swirl are licensed under MIT. See the License section


Examples

struct Size {
	width: u32,
	height: u32
}

#[coil::background_task]
async fn resize_image(id: u32, size: Size) -> Result<(), Error> {
	// some work
}

With an environment

struct Size {
	width: u32,
	height: u32
}

struct Environment {
    file_server_private_key: String,
    http_client: http_lib::Client,
    conn: sqlx::PgPool
}

#[coil::background_task]
async fn resize_image(env: &Environment, id: u32, size: Size) -> Result<(), Error> {
	// some work
}
resize_image_with_env("tohru".to_string(), Size { height: 32, width: 32 }).enqueue(&pool).await;
let runner = coil::RunnerBuilder::new(env, Executor, pool)
    .num_threads(8)
    .build()
    .unwrap();
runner.run_all_pending_tasks().await.unwrap()

Differences from swirl

  • Supports asynchronous jobs/executors
  • Supports jobs with generic arguments
  • Serializes data into Postgres with Messagepack instead of JSON
  • In asynchronous jobs, database queries will be run asynchronously with SQLx
  • Migrations are included in the binary and exposed via a migrate fn.
  • Enqueue is an async fn

License

This program includes code from the Swirl library, used under the MIT License or https://opensource.org/licenses/MIT

This program is sublicensed under GPLv3. An original MIT license copy for Swirl is provided in the source.

Dependencies

~21MB
~502K SLoC