1 unstable release
0.2.1 | Aug 5, 2024 |
---|
#528 in HTTP server
46KB
560 lines
AWS Lambda Runtime Emulator for Local and Remote Debugging
This emulator allows running Lambda functions locally with either a local payload from a file or a remote payload from AWS as if the local lambda was running there.
Installation and usage
cargo install lambda-debugger
Debugging with local payload
Use this method for simple use cases where a single static payload is sufficient.
- Save your payload into a file, e.g. save
{"command": "echo"}
intotest-payload.json
file - Start the emulator with the payload file name as its only param, e.g.
cargo lambda-debugger test-payload.json
- Add env vars printed by the emulator and start your lambda with
cargo run
in a separate terminal
The lambda will connect to the emulator and receive the payload. You can re-run your lambda with the same payload as many times as needed.
Debugging with remote payload
Use this method to get dynamic payload from other AWS services or when you need to send back a dynamic response, e.g. to process a request triggered by a user action on a website involving API Gateway as in the following diagram:
Remote debugging configuration
This project provides the tools necessary to bring the AWS payload to your local machine in real-time, run the lambda and send back the response as if the local lambda was running on AWS.
- proxy-lambda forwards Lambda requests and responses between AWS and your development machine in real time
- lambda-debugger provides Lambda APIs to run a lambda function locally and exchange payloads with AWS
Limitations
This Lambda emulator does not provide the full runtime capabilities of AWS:
- no environment constraints, e.g. memory or execution time
- panics are not reported back to AWS
- no concurrent request handling
- no support for X-Trace or Extensions APIs
- no stream responses
- smaller maximum payload size
Getting started with remote debugging
- create request and response queues in SQS with IAM permissions
- deploy proxy-lambda in place of the function you need to debug
- run the emulator locally as a binary or with
cargo run
- run your lambda locally with
cargo run
SQS configuration
Create two SQS queues with an identical configuration:
proxy_lambda_req
for requests to be sent from AWS to your local lambda under debugging. Required.proxy_lambda_resp
if you want responses from your local lambda to be returned to the caller. Optional.
See Advanced setup section for more info on how to customize queue names and other settings.
Recommended queue settings:
- Queue type: Standard
- Maximum message size: 256 KB
- Default visibility timeout: 10 Seconds
- Message retention period: 1 Hour
- Receive message wait time: 20 Seconds
This IAM policy grants proxy-lambda access to the queues. It assumes that you already have sufficient privileges to access Lambda and SQS from your local machine.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::512295225992:role/lambda_basic"
},
"Action": [
"sqs:DeleteMessage",
"sqs:GetQueueAttributes",
"sqs:ReceiveMessage",
"sqs:SendMessage"
],
"Resource": "arn:aws:sqs:us-east-1:512295225992:proxy_lambda_req"
}
]
}
Modifying the policy with your IDs
You need to replace Principal and Resource IDs with your values before adding the above policy to your queues:
- Principal - is the IAM role your lambda assumes (check Lambda's Permissions Config tab in the AWS console to find the value)
- Resource - the ARN of the queue the policy is attached to (see the queue details page in the AWS console to find the value)
Use different Resource values for request and response queues:
arn:aws:sqs:[your_region]:[your_aws_account]:proxy_lambda_req
forproxy_lambda_req
queuearn:aws:sqs:[your_region]:[your_aws_account]:proxy_lambda_resp
forproxy_lambda_resp
queue
Building and deploying proxy-lambda
The proxy-lambda function should be deployed to AWS in place of the function you want to debug.
Replace the following parts of the bash script below with your values before running it from the project root:
- target - the architecture of the lambda function on AWS, e.g.
x86_64-unknown-linux-gnu
- region - the region of the lambda function, e.g.
us-east-1
- name - the name of the lambda function you want to replace with the proxy, e.g.
my-lambda
target=x86_64-unknown-linux-gnu
region=us-east-1
name=my-lambda
cargo build --release --target $target
cp ./target/$target/release/proxy-lambda ./bootstrap && zip proxy.zip bootstrap && rm bootstrap
aws lambda update-function-code --region $region --function-name $name --zip-file fileb://proxy.zip
A deployed proxy-lambda should return OK or time out waiting for a response if you run it with a test event from the AWS console. Check CloudWatch logs for a detailed execution report.
Debugging
Pre-requisites:
- proxy-lambda was deployed to AWS
- SQS queues were created with the recommended access policies
Launching the local lambda:
- run
cargo lambda-debugger
in a separate terminal - add environmental variables from the prompt printed by the emulator
- start your lambda with
cargo run
in the same terminal where you added the env vars
Debugging:
- trigger the event on AWS as part of your normal data flow, e.g. by a user action on a webpage
- the emulator should display the lambda payload and forward it to your local lambda for processing
- debug as needed
Success, failure and replay:
- successful responses are sent back to the caller if the response queue is configured (
proxy_lambda_resp
) - panics or handler errors are not sent back to AWS
- the same incoming SQS message is reused until the lambda completes successfully
- lambda-debugger deletes the request message from
proxy_lambda_req
queue when the local lambda completes successfully - proxy-lambda deletes the response message from
proxy_lambda_resp
queue after forwarding it to the caller, e.g. to API Gateway - proxy-lambda purges
proxy_lambda_resp
queue before sending a new request toproxy_lambda_resp
- you have to purge
proxy_lambda_req
queue manually to delete stale requests
If the local lambda fails, terminates or panics, you can make changes to its code and run it again to reuse the same incoming payload from the request queue.
Advanced remote debugging setup
Custom SQS queue names
By default, proxy-lambda and the local lambda-debugger attempt to connect to proxy_lambda_req
and proxy_lambda_resp
queues in the same region.
Provide these env vars to proxy-lambda and lambda-debugger if your queue names differ from the defaults:
PROXY_LAMBDA_REQ_QUEUE_URL
- request queue, e.g. https://sqs.us-east-1.amazonaws.com/512295225992/debug_requestPROXY_LAMBDA_RESP_QUEUE_URL
- response queue, e.g. https://sqs.us-east-1.amazonaws.com/512295225992/debug_response
Late responses
Debugging the local lambda may take longer than the AWS service is willing to wait. For example, proxy-lambda function can be configured to wait for up to 15 minutes, but the AWS API Gateway wait time is limited to 30 seconds.
Assume that it took you 5 minutes to fix the lambda code and return the correct response. If proxy-lambda was configured to wait for that long it would still forward the response to the API Gateway which timed out 4.5 min earlier. In that case, you may need to trigger another request for it to complete successfully end-to-end.
Not waiting for responses from local lambda
It may be inefficient to have proxy-lambda waiting for a response from the local lambda because it takes too long or no response is necessary. Both proxy-lambda and lambda-debugger would not expect a response if the response queue is inaccessible.
Option 1: delete proxy_lambda_resp queue
Option 2: add PROXY_LAMBDA_RESP_QUEUE_URL
env var with no value to proxy-lambda and lambda-debugger
Option 3: make proxy_lambda_resp queue inaccessible by changing its IAM policy.
E.g. change the resource name from the correct queue name "Resource": "arn:aws:sqs:us-east-1:512295225992:proxy_lambda_resp"
to a non-existent name like this "Resource": "arn:aws:sqs:us-east-1:512295225992:proxy_lambda_resp_BLOCKED"
.
Both proxy-lambda and lambda-debugger treat the access error as a hint to not expect a response.
Canceling long proxy-lambda wait
If your proxy-lambda is configured to expect a long debugging time, e.g. 30 minutes, you may want to cancel the wait for a rerun.
Since it is impossible to kill a running lambda instance on AWS, the easiest way to cancel the wait is to send a random message to proxy_lambda_resp
queue via the AWS console.
The waiting proxy-lambda will forward it to the caller and become available for a new request.
Large payloads and data compression
The size of the SQS payload is limited to 262,144 bytes by SQS while Lambda allows up to 6MB. proxy-lambda and lambda-debugger compress oversized payloads using flate2 crate and send them as an encoded Base58 string to get around that limitation.
The data compression can take up to a minute in debug mode. It is significantly faster with release builds.
Logging
Both proxy-lambda and lambda-debugger use RUST_LOG
env var to set the logging level and filters.
If RUST_LOG
is not present or is empty, both crates log at the INFO level and suppress logging from their dependencies.
See https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#example-syntax for more info.
Examples of RUST_LOG
values:
error
- log errors only from all crates and dependencieswarn,lambda_debugger=info
- INFO level for the lambda-debugger, WARN level for everything elseproxy=debug
- detailed logging in proxy-lambda
Dependencies
~21–32MB
~466K SLoC