Idempotent-resource-ops

A set of functions to provide stateless idempotent requests based on resource metadata verification

Goal

• Stateless idempotency.
• Generic, usable with any HTTP server framework or date-time library.
• Customizable.

Usage

In the docs.rs is described how this lib implements idempotency, tips for API design when using this approach, customization guide and default parameters for common use cases

lib.rs:

Implementation of resource verification based on the HTTP Conditional headers and other custom headers to provide stateless idempotent requests.

How does idempotency is achieved?

The request idempotency is achieved by using the conditional headers If-None-Match, If-Match and If-Unmodified-Since to avoid common problems mutating resources, like the lost update, and ensuring that the request is only processed once.

With the conditional headers, the server state can be preserved in case of duplicated request, althoght, a client that encounters a timeout error and make a retry receiving a precondition failed cannot easily know if the server already processed the request.

Accept-Modified-Since header

For a server and client consensus, a custom header Accept-Modified-Since is supported, verfing if the request could be a retry of a call that already modified the resource in case the coditional header does not match. This header can be used by the client in every request that could accept a resource modified after the prevoius potential update.

Usage

In every request the client issues, Accept-Modified-Since header should be set to the first time the client is sending the request, so in case the response does not arrive, a retry could be send with Accept-Modified-Since as the same as the first attempt.

Make the first attempt in Tue, 13 Dec 2022 17:08:09

PUT /blog/post HTTP/1.1
If-Match: #82c8d70155524b43a5652e04ad27cadf~V0000000000000003
Accept-Modified-Since: 2022-12-13T17:08:09.141Z
...

Timeout or some network error

HTTP/1.1 408 Request Timeout
Date: Tue, 13 Dec 2022 17:08:10 GMT
...

Retry the request after some time with same Accept-Modified-Since header

PUT /blog/post HTTP/1.1
If-Match: #82c8d70155524b43a5652e04ad27cadf~V0000000000000003
Accept-Modified-Since: 2022-12-13T17:08:09.141Z
...

By using Accept-Modified-Since with the first attempt date, the server can verify if the resource was already modified, returning a success response.

Safety

However, in case the first request is not processed and a concurrent update is made in the resource, retring with Accept-Modified-Since will cause the server to accept the retry not processing the request and returning a success response.

Although the header semantics still preserved, if a request must be guaranteed to be applied, the Accept-Modified-Since is not recommended.

Creating resources

Create operations are not idempotent calls by default, but when some rules are applied to these operations they can become idempotent. For example, pushing a value in a Vec vs. inserting a value in a Set.

let mut vec = Vec::new();
vec.push(1);
vec.push(2);
vec.push(2);

assert_eq!(vec.len(), 3);

let mut set = HashSet::new();
set.insert(1);
set.insert(2);
set.insert(2);

assert_eq!(set.len(), 2);

Based on this example, using unique value for all resources of the same type can make a create operation idempotent.

Therefore, by generating resource IDs in the client, a create operation is guaranteed to produce only one resource if repeated multiple times.

ID colisions

Creating resources with client generated IDs can possibly lead to colisions. For that reason, always validate if there is a resource with the same ID is a behaviour enforced by the If-None-Match: * header.

However, with Accept-Modified-Since, a request could misuse the header to allow resources created with a greater date range.

PUT /blog/post HTTP/1.1
If-Match: #82c8d70155524b43a5652e04ad27cadf~V0000000000000003
Accept-Modified-Since: 2000-12-13T17:08:09.141Z

In order to prevent this exploit, the Accept-Modified-Since date can be lower bound to the moment the request was made minus some duration allowed to verify possible retries ($time::now() - ACCEPT_MODIFIED_SINCE_DURATION_LIMIT), resulting in a limited range of resources that may have a ID colision with the one being created.

Choosing a duration for ACCEPT_MODIFIED_SINCE_DURATION_LIMIT should consider the ID entropy and how many resources are created through the entire duration. As a safe configuration, a UUID v4 and a duration of 24 hours should be a good standard for most of the use cases.

HTTP create request

In HTTP APIs there are 2 possible ways to implement create semantics:

• POST
• PUT with If-None-Match

Bouth of then need to pass the resource ID in the request to be idempotent, however, to favor standardization of API endpoints, PUT with If-None-Match header could be prefered for idempotent calls and POST for non-idempotent calls.