3 releases (breaking)
Uses old Rust 2015
0.4.0 | Feb 24, 2017 |
---|---|
0.2.0 | Feb 19, 2017 |
0.1.0 | Feb 18, 2017 |
#1193 in Concurrency
15KB
374 lines
monotone
Counters and queues for configuration management in distributed systems
Monotone is a library and cli for maintaining atomic counters and queues. The implementations are designed with configuration management in mind. Note the counters are not performance counters for use in event tracking.
Two implementations are include:
- a single-process implementation, synchronised by
Arc<Mutex<...>>
- a DynamoDb implementation which uses conditional updates for optimistic locking
Documentation
Building / Installing
The source repository contains two projects. The library is called monotone, and the cli lives in a folder called cli, although it builds an executable file called monotone.
Library
You can link to the library by adding it as a dependency to your Cargo.toml as usual.
Select the aws
feature to bring in rusoto and use the DynamoDb backend.
The default is just the in-memory backend.
[dependencies]
monotone = { version = "0.4", features = ["aws"] }
CLI on Laptop / Development env
Install rust. Stable rust is fine, but it should be at least 1.15.
Consider using rustup: https://www.rustup.rs
Then run cargo install monotone
to install the version from https://crates.io
or cargo install --path=.
to install directly from checked out source.
CLI on CI / CD
Either install the rust toolchain on your jenkins or use a docker container like this one: https://hub.docker.com/r/jimmycuadra/rust/
Then build as you would in your dev env and copy the built artifact somewhere safe.
Testing
The monotone/tests
folder contains integration tests.
The terraform
folder contains infrastructure definitions for running the integration tests. See the readme file there.
Cli commands
Each counter or queue is stored in its own row in the table in DynamoDb.
The -i
parameter selects which row.
The cli will prevent you running counter commands on a queue and visa versa.
Counter
Counter is a simple atomic counter. Run like so:
monotone -i mycounter counter get
will return
{
"id": "mycounter",
"value": 0,
"region": "eu-west-1",
"table": "Counters"
}
Increment the counter like so:
monotone -i mycounter counter next
will return
{
"id": "mycounter",
"value": 1,
"region": "eu-west-1",
"table": "Counters"
}
Queue
The queue is a list of string process IDs. Each entry in the queue is given the monotonic counter value when it joins the list. The list is sorted in ascending order of counter value.
The queue contains a fencing token which is returned by all operations. This will monotonically increase with every write to storage. Use this for conditional updates in other systems to prevent acting on a stale view of the queue.
Add a process ID to the queue like so:
monotone -i myqueue queue -p foo join
which will output something like this:
{
"id": "myqueue",
"region": "eu-west-1",
"table": "Counters",
"fencing_token": 1,
"ticket": {
"process_id": "foo",
"counter": 1,
"position": 0
}
}
You can also remove a node (for tidyness) like so:
monotone -i myqueue queue -p foo leave
which prints output:
{
"id": "myqueue",
"region": "eu-west-1",
"table": "Counters",
"fencing_token": 2
}
To list the nodes use:
monotone -i myqueue queue list
which will output something like this:
{
"id": "myqueue",
"region": "eu-west-1",
"table": "Counters",
"fencing_token": 2,
"tickets": [
{
"process_id": "foo",
"counter": 1,
"position": 0
}
]
}
Example Usecases
Assigning server IDs to nodes in a Zookeeper cluster
Zookeeper is a good place to store atomic counters like the ones implemented in this crate. But what if you don't have a zookeeper cluster yet and you're trying to build one? You have to build on something you do have, like DynamoDb.
On first boot, run the cli's queue command like so (make very sure your hostnames are unique e.g. EC2 instance IDs!):
monotone -i myzkcluster queue -p $(hostname -f) join | jq .ticket.counter
Write the resulting value to /etc/zookeeper/conf/myid
as appropriate.
Note the zookeeper docs say the server ID must be between 0 and 255. Monotone uses the full range of u64 integers.
Simple leader election or lock
The list of nodes in the queue is enough to nominate a distinguished process or leader.
Just use the first process in the queue.
There are several limitations:
- There's no liveness checking to remove failed processes from the queue
- You must use the fencing token to ensure the queue hasn't changed while acting as leader / holding the lock.
Dependencies
~21–32MB
~627K SLoC