4 releases
0.2.1 | Apr 16, 2024 |
---|---|
0.2.0 | Apr 16, 2024 |
0.1.1 | Apr 4, 2024 |
0.1.0 | Apr 2, 2024 |
#1641 in Network programming
608 downloads per month
35KB
663 lines
AWS Lambda Log Proxy
Filter or transform logs from AWS Lambda functions before they are sent to CloudWatch Logs.
[!NOTE] This is a library for developers. If you are looking for a binary executable, see AWS Lambda Log Filter.
[!CAUTION] Possible data loss if you write tons of logs and return immediately. See possible data loss below.
Usage
Installation
Add the following to the dependencies
in your Cargo.toml
:
aws-lambda-log-proxy = "0.2"
or run:
cargo add aws-lambda-log-proxy
Examples
A real world case: AWS Lambda Log Filter.
Documentation
FAQ
Why We Need This?
We need a solution to realize the following features without modifying the existing code of AWS Lambda functions:
- Reduce the volume of logs to lower the costs.
- Wrap existing logs in JSON with customizable level and field name, so we can use the built-in Lambda Log-Level Filtering to filter them.
- But keep the EMF logs untouched so we can still retrieve the metrics.
How It Works?
We use AWS Lambda Runtime Proxy to spawn the handler process as a child process of the proxy (and optionally disable the _LAMBDA_TELEMETRY_LOG_FD
environment variable) to intercept the logs, then suppress the invocation/next
request to process logs as much as possible.
Why Not Just Grep
Actually you could just create a shell scripts with the content exec "$@" 2>&1 | grep --line-buffered xxx
and use the lambda runtime wrapper script to realize a simple filter. However we need this lib to realize the following features:
- Override the
_LAMBDA_TELEMETRY_LOG_FD
environment variable to ensure the logs are printed to the stdout/stderr instead of the telemetry log file. - Suppress the
invocation/next
request to process logs as much as possible.
Possible Data Loss
Though we tried our best to suppress the invocation/next
request to process logs as much as possible, if you write tons of logs (more than thousands of lines) and return immediately, there might be some logs not processed.
As a best practice, it is your responsibility to do a thorough benchmark test against your own use case to ensure the logs are processed as expected.
CHANGELOG
Dependencies
~6–16MB
~185K SLoC