5 unstable releases

0.4.0 Jun 15, 2021
0.2.2 Feb 8, 2021
0.2.1 Feb 8, 2021
0.2.0 Sep 3, 2020
0.1.0 Aug 18, 2020

#1047 in Database interfaces

GPL-3.0 license

42KB
841 lines

Coil

An async task queue built with SQLx, Postgres, and Rayon

Coverage Crates.io Crates.io

Coil is heavily inspired by and takes heavily from swirl. In many places of the codebase, code is very similiar.

Supports synchronous and asynchronous jobs. Synchronous jobs will be spawned into a threadpool managed by rayon. Async jobs will be spawned onto an executor. The only requirement is that the executor implements the futures Spawn trait. This way, coil supports Tokio, smol, and async-std.

† This software is alpha, and not intended for production use yet. Use at your own risk.

†† This software is sublicensed as GPLv3. Portions from swirl are licensed under MIT. See the License section


Examples

struct Size {
	width: u32,
	height: u32
}

#[coil::background_task]
async fn resize_image(id: u32, size: Size) -> Result<(), Error> {
	// some work
}

With an environment

struct Size {
	width: u32,
	height: u32
}

struct Environment {
    file_server_private_key: String,
    http_client: http_lib::Client,
    conn: sqlx::PgPool
}

#[coil::background_task]
async fn resize_image(env: &Environment, id: u32, size: Size) -> Result<(), Error> {
	// some work
}
resize_image_with_env("tohru".to_string(), Size { height: 32, width: 32 }).enqueue(&pool).await;
let runner = coil::RunnerBuilder::new(env, Executor, pool)
    .num_threads(8)
    .build()
    .unwrap();
runner.run_all_pending_tasks().await.unwrap()

Differences from swirl

  • Supports asynchronous jobs/executors
  • Supports jobs with generic arguments
  • Serializes data into Postgres with Messagepack instead of JSON
  • In asynchronous jobs, database queries will be run asynchronously with SQLx
  • Migrations are included in the binary and exposed via a migrate fn.
  • Enqueue is an async fn

License

This program includes code from the Swirl library, used under the MIT License or https://opensource.org/licenses/MIT

This program is sublicensed under GPLv3. An original MIT license copy for Swirl is provided in the source.


lib.rs:

coil is a efficient background job queue for Postgres. The API is very similiar and indeed based upon swirl. In addition to the functionality swirl offers, however, coil can:

  • Queue asynchronous tasks for execution on an executor, whether it be smol, tokio or async-std
  • Queue functions with generics
  • SQL queries in coil are ran asynchronously wherever possible
  • Migrations are stored in the binary, and accessible via a migrate() fn. No more needing to copy-paste migration files!

Dependencies

~24MB
~527K SLoC