LIVE atabany.net

AWS Serverless Platform

A production-pattern serverless event ingestion pipeline built entirely on AWS and managed with Terraform. API Gateway receives HTTP POST requests, Lambda processes and enriches them, S3 stores structured JSON events. Every resource is compliance-tagged, OIDC-authenticated CI/CD deploys with zero static credentials. The live demo below hits the real endpoint.

AWS Lambda · Python 3.12 API Gateway v2 · HTTP S3 · AES256 · versioned Terraform 1.14 · 5 modules S3 remote state · DynamoDB lock GitHub Actions · OIDC · zero keys CloudWatch alarms · log retention
View on GitHub → Live demo

Pipeline Overview

The pipeline is intentionally serverless — no EC2 instances, no idle compute costs. An HTTP API Gateway endpoint (POST /ingest) is the only public surface. It invokes a Lambda function via AWS_PROXY integration, which validates the payload, attaches a UUID and UTC timestamp, then writes a structured JSON object to S3 partitioned by date (events/YYYY-MM-DD/uuid.json). All within free tier at this traffic level.

Throttling is enforced at the API Gateway stage: 10 requests/second rate, 20 burst — enough for any legitimate demo load, prevents abuse. CloudWatch monitors Lambda error rate and duration, alarming if errors occur or if average duration exceeds 2000ms (1 second below the 3s timeout).

AWS serverless ingestion pipeline architecture AWS SERVERLESS PIPELINE // eu-central-1 HTTP Client POST /ingest JSON payload API GATEWAY v2 HTTP API 10 rps throttle AWS_PROXY integration LAMBDA ingest.handler Python 3.12 · 128MB UUID + timestamp + write S3 DATA LAKE portfolio-platform events/YYYY-MM-DD/ AES256 · versioned · private CLOUDWATCH Logs + Alarms error rate · 14d retention IAM ROLE Least privilege PutObject · GetObject only GITHUB ACTIONS OIDC Deploy zero static keys push → zip → deploy Lambda TERRAFORM 5 modules · S3 state storage · iam · lambda · apigw · observability // Every resource tagged: Project · Owner · Environment · ManagedBy · Repository
Full pipeline from HTTP client to S3 storage. CloudWatch monitors Lambda. GitHub Actions deploys via OIDC. Terraform manages all five resource groups as independent modules with shared compliance tags.

Infrastructure as Code

The project uses modular Terraform — each concern is a separate module with its own variables.tf, main.tf, and outputs.tf. Modules communicate only through explicit output → variable wiring in the root module. No module references another's resources directly. Remote state in S3 with DynamoDB locking prevents concurrent apply corruption.

MODULE RESOURCES WHY SEPARATE
storage aws_s3_bucket, versioning, encryption, public_access_block Data layer — other modules receive the bucket ARN/ID as inputs, never reference the resource directly. Keeps storage lifecycle independent.
iam Lambda exec role, S3 write policy, CloudWatch logs policy, OIDC provider, GitHub Actions role All identity and permissions in one place. The OIDC provider and GitHub role are co-located here because they're both identity concerns, not deployment concerns.
lambda aws_lambda_function, archive_file (zip packaging) Application code boundary. The archive provider zips ingest.py and source_code_hash forces redeployment on code changes automatically.
apigateway HTTP API, default stage (throttled), Lambda integration, POST /ingest route, Lambda invoke permission Public surface — isolated so throttling, routing, and CORS can be modified without touching the Lambda or IAM modules.
observability CloudWatch log group (14d retention), error rate alarm, duration alarm Observability as a first-class concern, not an afterthought. Importing the pre-existing log group into state (rather than recreating it) was an intentional migration step.

Zero Static Credentials

GitHub Actions authenticates to AWS using OIDC federation — no AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY stored anywhere in GitHub. When a workflow runs, GitHub presents a signed JWT to AWS STS. AWS verifies the token against the registered OIDC provider and issues temporary credentials scoped to the portfolio-platform-dev-github-actions IAM role.

The role's trust policy uses a StringLike condition on the token.actions.githubusercontent.com:sub claim, locking it to repo:omaratabany/portfolio-aws-platform:*. A fork of the repo cannot assume this role. Credentials are valid only for the duration of the job — typically under 2 minutes. The deployment workflow only triggers on pushes to main that change files under functions/, avoiding unnecessary deploys on documentation or Terraform-only changes.

OIDC FLOW
git push → main GitHub Actions JWT to AWS STS Temp credentials (role) zip + deploy Lambda

Credentials expire when the job ends. No rotation required. No secret sprawl.

Hit the Real Endpoint

The form below sends a real POST request through this Cloudflare Worker (as a proxy) to the AWS API Gateway endpoint in eu-central-1. Lambda processes it, writes a JSON event to S3, and returns the event ID and S3 key. You're invoking a live serverless pipeline.

LIVE · portfolio-platform-dev-ingest · eu-central-1
POST /ingest · 10 rps throttle

Enter a message and send it. A real AWS Lambda function will receive it, add a UUID and timestamp, write it to S3, and return the storage key.

// Rate limited to 10 req/s · payload capped at 200 chars · proxied via Cloudflare Worker to avoid CORS