#aws #s3

bin+lib esthri

Extremely simple (memory stable) S3 client that supports get, put, head, list, and sync

6 stable releases

new 7.0.0 Jan 11, 2022
6.3.0 Dec 7, 2021
6.2.0 Dec 2, 2021
6.1.2 Nov 3, 2021
6.1.1 Oct 28, 2021

#422 in Command line utilities

Download history 40/week @ 2021-10-25 95/week @ 2021-11-01 81/week @ 2021-11-08 82/week @ 2021-11-15 48/week @ 2021-11-22 146/week @ 2021-11-29 48/week @ 2021-12-06 16/week @ 2021-12-13 10/week @ 2021-12-20 8/week @ 2021-12-27 27/week @ 2022-01-03 62/week @ 2022-01-10

107 downloads per month

MIT license

145KB
3.5K SLoC

Esthri (S3)

Build Status

Extremely simple (memory stable) S3 client that supports get, put, head, list, and sync.

esthri 6.3.0
Simple S3 file transfer utility.

USAGE:
    esthri <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    abort           Manually abort a multipart upload
    etag            Compute and print the S3 ETag of the file
    get             Download an object from S3
    head-object     Retreive the ETag for a remote object
    help            Prints this message or the help of the given subcommand(s)
    list-objects    List remote objects in S3
    put             Upload an object to S3
    serve           Launch an HTTP server attached to the specified bucket
    sync            Sync a directory with S3

AWS S3 Compatibility Layer

The esthri CLI tool additionally provides a AWS CLI compatibility mode where it is able to handle cp and sync operations with an identical CLI interface as the aws s3 tool. Currently, only the basic case is implemented for these commands and only some optional arguments are handled.

To use, the esthri binary must either be:

  • Named or hard-linked as aws
  • Run with the ESTHRI_AWS_COMPAT_MODE environment variable set

In this mode, esthri will attempt to transparently invoke the real aws tool if it encounters a command it cannot handle. For this to work, the path to the real aws tool should be put in an environment variable named ESTHRI_AWS_PATH. For example: ETHRI_AWS_PATH=/usr/bin/aws. If this environment variable is not specified then esthri will default to invoking aws.real as the aws tool.

$ ln -s /usr/local/bin/aws /usr/local/bin/aws.real
$ ESTHRI_AWS_COMPAT_MODE=1 esthri s3 help # prints the S3 help text, as generated by the aws command

Transparent Sync Compression

esthri can transparently compress files such that they are compressed within S3 but not on the local filesystem. To enable, either set the ESTHRI_AWS_COMPAT_MODE_COMPRESSION environment variable or use the --transparent-compression CLI option. For example:

ESTHRI_AWS_COMPAT_MODE=1 esthri s3 sync mydirectory/ s3://esthri-test/myfiles/ --transparent-compression # syncs as normal, however files in S3 are gzipped
ESTHRI_AWS_COMPAT_MODE=1 esthri s3 sync s3://esthri-test/myfiles/ mynewdirectory/ --transparent-compression # syncs and decompresses files as they are synced down. mydirectory and mynewdirectory will now contain the same files

esthri makes use of S3 metadata to do this transparent compression, storing the esthri version number in the metadata of files it has compressed. For this reason, the decompression will only work for files that esthri has compressed itself.

Releasing

cargo-release is used to automate releases of esthri using the release.toml file for release configuration.

To perform a release, run cargo release <release-level> -x.

Copyright

Copyright (C) 2021 Swift Navigation Inc.
Contact: Swift Navigation <dev@swiftnav.com>

This source is subject to the license found in the file 'LICENSE' which must be
be distributed together with this source. All other rights reserved.

THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND,
EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE.

Dependencies

~10–17MB
~322K SLoC