s3-bucket-stats

Current version: 0.8.0

Prometheus metrics endpoint for S3 bucket stats.

Modes

This tool can be run in either single or serve mode.

In single mode, Prometheus metrics will be generated and output to stdout.

In serve mode, a simple HTTP server will be started at --listen-address and --listen-port with a single endpoint at /metrics. TLS encryption or any type of authentication is not supported.

Configuration

Configuration can be done with command line arguments. To pass S3 credentials and configuration, use the common AWS Environment variables. You'll probably at least want to specify:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_ENDPOINT_URL
• AWS_REGION

For a full list of supported AWS variables, see the source of reqsign/services/aws-v4 .

All CLI arguments can also be specified als environment variable. Do note that METRICS_EXTRA_LABELS is limited to a single value, if multiple values are required then specify --metrics-extra-labels multiple times. See the output of --help for the available names.

Breaking changes

With version 0.8.0 a switch was made to another S3 library, and some AWS environment variable names were changed. In particular: AWS_ENDPOINT to AWS_ENDPOINT_URL and AWS_DEFAULT_REGION to AWS_REGION. AWS_REGION is now mandatory.

Kubernetes

This tool was created to be run as a service in container, for easy scraping using a ServiceMonitor or PodMonitor. In serve mode there is a single HTTP endpoint at /metrics that will contain all Prometheus metrics. This endpoint will return a 404 error until the metrics have been loaded for the first time. For buckets with a large amount of objects this can take several minutes, so take care when using this endpoint for liveness or readiness checks.

There are four separate endpoints for health probes: /healthz, /healthz/startup, /healthz/liveness and /healthz/readiness. The first three will return 200 OK as soon as the HTTP daemon is running, and until the expiration_interval has expired. If the metrics have been marked as stale, a 503 SERVICE UNAVAILABLE will be returned. The readiness endpoint will return 503 SERVICE UNAVAILABLE until the metrics have been loaded, then it will return 200 OK.

Deployment

A Helm chart is available in the charts/ subdirectory and in the Gitlab Helm registry.

License

GPL-3.0-only, see LICENSE file for details.