HTTP Tunnel

🌐 English | 中文文档

A serverless HTTP tunnel built with Rust and AWS Lambda, providing secure access to local development servers through public URLs - similar to ngrok, but fully serverless and self-hosted.

English

Overview

HTTP Tunnel allows you to expose local services (like localhost:3000) to the internet through a public URL. Perfect for:

• Testing webhooks during local development (Stripe, GitHub, Twilio, etc.)
• Sharing work-in-progress with clients or teammates
• Testing mobile apps against local backends
• Demoing features without deploying
• API development with external services requiring public URLs
• IoT development with callback testing

Architecture: Fully serverless (AWS Lambda + API Gateway + DynamoDB) for cost-effective, auto-scaling infrastructure with zero operational overhead.

Features

• Serverless Architecture: Zero operational overhead, pay only for actual usage
• Secure WebSocket Tunneling: Encrypted persistent connections (WSS/HTTPS)
• Automatic Reconnection: Exponential backoff strategy handles network interruptions gracefully
• JWT/JWKS Authentication: Optional token-based authentication with RSA/HMAC support
• Custom Domains: Support for custom domain names with ACM certificates
• Fast & Efficient: Low-latency request forwarding powered by Rust performance
• Event-Driven: Optional DynamoDB Streams + EventBridge for optimized response delivery
• Load Testing Ready: Handles concurrent requests with proper timeout handling
• Multiple HTTP Methods: Full support for GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS
• Binary Data Support: Base64 encoding for binary request/response bodies
• Open Source: MIT licensed, fully customizable and auditable

Quick Start

Prerequisites

• Rust 1.70+ with cargo
• AWS account with configured credentials
• Node.js 18+ (for infrastructure deployment)
• Pulumi CLI (for infrastructure management)

Installation

Option 1: Build from source

# Clone the repository
git clone https://github.com/tyrchen/http-tunnel.git
cd http-tunnel

# Build the forwarder agent
cargo build --release --bin ttf

# The binary will be at target/release/ttf

Option 2: Install via cargo

cargo install --git https://github.com/tyrchen/http-tunnel --bin ttf

Deploy Infrastructure

# Deploy AWS infrastructure (Lambda, API Gateway, DynamoDB)
make deploy-infra

# Note the WebSocket endpoint from the output

Start Forwarding

# Forward local service to the internet (uses default endpoint and port 3000)
ttf

# Or specify custom endpoint and port
ttf --endpoint wss://YOUR_WEBSOCKET_ENDPOINT --local-port 8080

# You'll receive a public URL like: https://abc123.execute-api.us-west-2.amazonaws.com

Now any HTTP request to your public URL will be forwarded to your local service.

Default Configuration:

• Endpoint: wss://ws.example.com/dev
• Local port: 3000
• Local host: 127.0.0.1

Architecture

Internet → API Gateway HTTP → Lambda → WebSocket → Local Agent → localhost:3000
                              ↓
                         DynamoDB (state)

Components:

• Local Forwarder (ttf): Rust CLI agent running on your machine
• Lambda Handler: Serverless functions for WebSocket and HTTP handling
• API Gateway: WebSocket API for agent connections, HTTP API for public requests
• DynamoDB: Tracks connections and pending requests

For detailed architecture, see specs/0001-idea.md.

Usage

Basic Usage

# Forward localhost:3000 to the internet (uses default endpoint)
ttf

# Or specify custom endpoint
ttf --endpoint wss://YOUR_ENDPOINT

With Custom Port

# Forward a different local port
ttf --port 8080

# Or use short form
ttf -p 8080

With Custom Domain

# Use your own domain (requires custom domain setup)
ttf --endpoint wss://ws.yourdomain.com

Environment Variables

# Override default endpoint via environment variable
export TUNNEL_ENDPOINT=wss://YOUR_CUSTOM_ENDPOINT

# Set authentication token
export TUNNEL_TOKEN=your_jwt_token

# Run with environment configuration
ttf

Project Structure

http-tunnel/
├── apps/
│   ├── forwarder/          # Local agent CLI (ttf binary)
│   └── handler/            # AWS Lambda function
├── crates/
│   └── common/             # Shared library (protocol, models, utilities)
├── infra/                  # Pulumi infrastructure as code
│   ├── src/                # TypeScript infrastructure modules
│   ├── scripts/            # Deployment helper scripts
│   └── README.md           # Infrastructure documentation
├── testapp/                # Example TodoMVC API server for testing
│   ├── main.py             # FastAPI application
│   └── pyproject.toml      # Python dependencies
└── specs/                  # Architecture and implementation specs
    ├── 0001-idea.md        # Architecture design
    ├── 0002-common.md      # Common library spec
    ├── 0003-forwarder.md   # Forwarder agent spec
    ├── 0004-lambda.md      # Lambda functions spec
    ├── 0005-iac.md         # Infrastructure spec
    └── 0006-implementation-plan.md

Development

Build Commands

# Build all components
cargo build

# Build forwarder agent only
cargo build --bin ttf

# Build Lambda handler (requires cargo-lambda)
cargo lambda build --release --arm64 --bin handler

# Run tests
cargo test

# Run linter
cargo clippy

Test Application

A sample TodoMVC API server is included in testapp/ for testing the HTTP tunnel:

# Run the test app on port 3000
make run-testapp

# The API will be available at http://localhost:3000
# Interactive docs at http://localhost:3000/docs

Test App Features:

• In-memory CRUD API for todo items
• Pre-loaded with meaningful dummy data
• RESTful endpoints: GET, POST, PUT, DELETE
• Perfect for testing the tunnel forwarding functionality

Example Usage:

# In terminal 1: Start the test app
make run-testapp

# In terminal 2: Start the tunnel forwarder (uses default endpoint and port 3000)
ttf

# In terminal 3: Access your local app via the public tunnel URL
curl https://YOUR_TUNNEL_URL/todos

Infrastructure Commands

# Preview infrastructure changes
make preview-infra

# Deploy infrastructure
make deploy-infra

# Destroy infrastructure
make destroy-infra

Custom Domain Setup

To use your own domain instead of API Gateway URLs:

1. Configure your domain in infra/Pulumi.dev.yaml
2. Set up ACM certificate (see infra/QUICKSTART_CUSTOM_DOMAIN.md)
3. Deploy infrastructure
4. Configure DNS records

See infra/README.md for detailed instructions.

How It Works

1. Agent Connection: The ttf CLI connects to AWS API Gateway WebSocket endpoint
2. Registration: Lambda assigns a unique subdomain/connection ID
3. HTTP Request: User makes HTTP request to the public URL
4. Forwarding: Lambda looks up the connection and forwards request via WebSocket
5. Local Processing: Agent receives request and forwards to local service
6. Response: Agent sends response back through WebSocket
7. Completion: Lambda receives response and returns to original HTTP caller

Configuration

Forwarder Configuration

ttf --help

Options:
  -e, --endpoint <URL>       WebSocket endpoint URL [default: wss://ws.example.com/dev]
  -p, --port <PORT>          Local service port to forward to [default: 3000]
  --host <HOST>              Local service host [default: 127.0.0.1]
  -t, --token <TOKEN>        Authentication token (JWT)
  -v, --verbose              Enable verbose logging
  --connect-timeout <SECS>   Connection timeout in seconds [default: 10]
  --request-timeout <SECS>   Request timeout in seconds [default: 25]

Environment Variables:

• TUNNEL_ENDPOINT: Override default WebSocket endpoint
• TUNNEL_TOKEN: Set authentication token

Infrastructure Configuration

Edit infra/Pulumi.dev.yaml:

config:
  aws:region: us-west-2
  aws:profile: your-aws-profile
  http-tunnel:environment: dev
  http-tunnel:lambdaArchitecture: arm64
  http-tunnel:lambdaMemorySize: "256"
  http-tunnel:lambdaTimeout: "30"
  http-tunnel:enableCustomDomain: "false"

See infra/README.md for all configuration options.

Cost Estimation

Approximate monthly costs (us-west-2 region):

AWS Free Tier may significantly reduce costs for development/testing usage.

Monitoring

The deployed infrastructure includes CloudWatch logs for:

• WebSocket connection events
• HTTP request forwarding
• Lambda function execution
• Error tracking

Access logs via AWS Console or CLI:

# View Lambda logs
aws logs tail /aws/lambda/http-tunnel-handler-dev --follow

# View API Gateway logs
aws logs tail /aws/apigateway/http-tunnel-dev --follow

Troubleshooting

Connection Issues

Problem: Agent can't connect to WebSocket endpoint

Solution:

1. Verify endpoint URL is correct (should start with wss://)
2. Check AWS credentials are configured
3. Ensure infrastructure is deployed (make deploy-infra)
4. Check CloudWatch logs for errors

Request Timeout

Problem: HTTP requests timeout waiting for response

Solution:

1. Ensure local service is running on specified port
2. Check agent is connected (should show "Connected" in logs)
3. Verify no firewall blocking local connections
4. Check Lambda timeout settings (increase if needed)

Custom Domain Not Working

Problem: Custom domain not resolving or returns errors

Solution:

1. Verify ACM certificate is in "ISSUED" status
2. Check DNS records are correctly configured
3. Wait 5-10 minutes for DNS propagation
4. See infra/CUSTOM_DOMAIN_SETUP.md for detailed troubleshooting

Documentation

• specs/README.md: Complete technical specifications
• specs/0001-idea.md: Architecture design document
• infra/README.md: Infrastructure deployment guide
• infra/QUICKSTART_CUSTOM_DOMAIN.md: Custom domain quick setup
• infra/CUSTOM_DOMAIN_SETUP.md: Complete custom domain reference

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass (cargo test)
6. Run linter (cargo clippy)
7. Submit a pull request

Comparison with ngrok

Security

• End-to-end TLS: All communication encrypted (HTTPS + WSS)
• Isolated Connections: Each connection has unique credentials
• No Persistent Storage: Request/response data not stored
• IAM Policies: Least-privilege access for Lambda functions
• TTL Cleanup: Automatic cleanup of stale data

For production use, consider:

• Implementing authentication on the WebSocket connection
• Adding request filtering/validation
• Setting up AWS WAF rules
• Enabling VPC endpoints for Lambda-DynamoDB communication

License

This project is distributed under the terms of MIT.

See LICENSE for details.

Copyright 2025 Tyr Chen

Acknowledgments

Inspired by ngrok and built with:

• Rust - Systems programming language
• Tokio - Async runtime
• AWS Lambda - Serverless compute
• Pulumi - Infrastructure as code