LIVE atabany.net

AWS Cloud Infrastructure Platform

Two independent pipelines in one Terraform-managed AWS account, both deployed via GitHub Actions with OIDC (zero static credentials anywhere): a serverless event-ingestion pipeline, and an uptime/status-monitor pipeline with its own alerting, CloudWatch alarms, and a read-only bridge into a self-hosted Grafana. Every phase is written up as an ADR in the repo — what was chosen, what was rejected, and why. The live demo below hits the real ingestion endpoint.

AWS Lambda · Python 3.12 API Gateway v2 · HTTP DynamoDB · S3 Terraform · 16+ modules EventBridge · SNS GitHub Actions · OIDC · zero keys Checkov · IAM Access Analyzer CloudWatch → Grafana bridge
View on GitHub → Live demo

Pipeline Overview

The pipeline is intentionally serverless: no EC2 instances, no idle compute costs. An HTTP API Gateway endpoint (POST /ingest) is the only public surface. It invokes a Lambda function via AWS_PROXY integration, which validates the payload, attaches a UUID and UTC timestamp, then writes a structured JSON object to S3 partitioned by date (events/YYYY-MM-DD/uuid.json). All within free tier at this traffic level.

Throttling is enforced at the API Gateway stage: 10 requests/second rate, 20 burst, enough for any legitimate demo load, prevents abuse. CloudWatch monitors Lambda error rate and duration, alarming if errors occur or if average duration exceeds 2000ms (1 second below the 3s timeout).

AWS serverless ingestion pipeline architecture AWS SERVERLESS PIPELINE // eu-central-1 HTTP Client POST /ingest JSON payload API GATEWAY v2 HTTP API 10 rps throttle AWS_PROXY integration LAMBDA ingest.handler Python 3.12 · 128MB UUID + timestamp + write S3 DATA LAKE portfolio-platform events/YYYY-MM-DD/ AES256 · versioned · private CLOUDWATCH Logs + Alarms error rate · 14d retention IAM ROLE Least privilege PutObject · GetObject only GITHUB ACTIONS OIDC Deploy zero static keys push → zip → deploy Lambda TERRAFORM 5 modules · S3 state storage · iam · lambda · apigw · observability // Every resource tagged: Project · Owner · Environment · ManagedBy · Repository
Full pipeline from HTTP client to S3 storage. CloudWatch monitors Lambda. GitHub Actions deploys via OIDC. Terraform manages all five resource groups as independent modules with shared compliance tags.

A Second, Independent Pipeline

A separate uptime monitor for atabany.net (and other targets), built as two independent loops that only share one DynamoDB table: a check loop (EventBridge on a 5-minute schedule → a checker Lambda → HTTPS request to each target → write to DynamoDB, alerting via SNS only when a target's status actually flips up↔down, not on every failed check during an outage), and a read loop (a static S3 status page → its own API Gateway → a read-only Lambda → DynamoDB). Neither loop calls the other directly.

The real difficulty here wasn't boilerplate CRUD — it was a genuine near-miss caught before it became a bill: DynamoDB's on-demand billing mode looks like the safer default for a low-traffic table, but AWS's always-free 25 RCU/25 WCU allowance applies specifically to provisioned capacity; on-demand has no equivalent ongoing free allowance outside a new account's first 12 months, and this account isn't new. Switched to provisioned (1 RCU/1 WCU, far more than the ~288 writes/day this needs) before that ever reached production.

Status monitor: independent check loop and read loop sharing one DynamoDB table STATUS MONITOR // TWO INDEPENDENT LOOPS, ONE TABLE CHECK LOOP // every 5 minutes EVENTBRIDGE rate(5 min) declarative schedule LAMBDA checker.py HTTPS check per target SNS on state change up↔down flips only Email alert one per flip, not per failure DYNAMODB UptimeChecks 1 RCU / 1 WCU provisioned write READ LOOP // on page load Visitor loads status page S3 STATIC SITE status page plain HTTP hosting API GATEWAY GET /status separate HTTP API LAMBDA api.py read-only role read // The two loops never call each other directly — DynamoDB is the only thing they share
Check loop (top) writes on a schedule and alerts on state change only. Read loop (bottom) serves the public status page. DynamoDB is the only thing they share.

Infrastructure as Code

The project uses modular Terraform: each concern is a separate module with its own variables.tf, main.tf, and outputs.tf. Modules communicate only through explicit output → variable wiring in the root module. No module references another's resources directly. Remote state in S3 with DynamoDB locking prevents concurrent apply corruption.

MODULE RESOURCES WHY SEPARATE
storage aws_s3_bucket, versioning, encryption, public_access_block Data layer. Other modules receive the bucket ARN/ID as inputs, never reference the resource directly. Keeps storage lifecycle independent.
iam Lambda exec role, S3 write policy, CloudWatch logs policy, OIDC provider, GitHub Actions role All identity and permissions in one place. The OIDC provider and GitHub role are co-located here because they're both identity concerns, not deployment concerns.
lambda aws_lambda_function, archive_file (zip packaging) Application code boundary. The archive provider zips ingest.py and source_code_hash forces redeployment on code changes automatically.
apigateway HTTP API, default stage (throttled), Lambda integration, POST /ingest route, Lambda invoke permission Public surface, isolated so throttling, routing, and CORS can be modified without touching the Lambda or IAM modules.
observability CloudWatch log group (14d retention), error rate alarm, duration alarm Observability as a first-class concern, not an afterthought. Importing the pre-existing log group into state (rather than recreating it) was an intentional migration step.

Auditing What Already Existed, Not Just What's New

A real finding, not a hypothetical one: the GitHub Actions deploy role — written before the status monitor existed — still granted lambda:UpdateFunctionCode on Resource: "*" (every function in the account) plus S3 permissions the deploy workflow never actually calls. A five-line grep confirmed it: those S3 actions had been unused since the policy was written. Scoped the role down to exactly the three function ARNs it deploys to, and dropped the S3 grant entirely. The monitored-site list also moved out of Lambda environment variables into SSM Parameter Store, so changing which sites get watched no longer requires a code redeploy.

The account-wide baseline added on top: IAM Access Analyzer (free, continuous, flags any resource policy granting access outside this account's trust zone), and a password policy (14-char minimum, all four character classes, 5-password reuse prevention) — deliberately without forced rotation. NIST 800-63B argues against mandatory periodic password rotation (it pushes people toward predictable incremented passwords); that's a documented decision, not an oversight the next scanner should flag. CloudTrail relies on the always-free 90-day default event history rather than a paid Trail, since a project with a $0.01-actual budget-alarm threshold shouldn't wave through "probably a fraction of a cent a month" as if it were free.

GuardDuty, Security Hub, and AWS Config are explicitly out of scope — all three bill per-event, per-check, or per-resource, a real ongoing cost against this project's $0 ceiling, not a rounding error. Naming that trade-off out loud, with the actual cost reason, is itself the practice worth demonstrating: "what would you add if the budget changed" is a real interview question, and a deferred control with a documented reason is a different thing entirely from a gap nobody noticed.

The credential-report audit this baseline enabled also surfaced two real, still-open gaps on the account's day-to-day IAM user: no MFA enabled, and an access key past its 90-day rotation point. Both are flagged here rather than silently fixed — MFA enrollment needs a physical device, and this project's own rule was to defer rotation until explicitly raised, not to rotate credentials out from under an account automatically.

Zero Static Credentials

GitHub Actions authenticates to AWS using OIDC federation: no AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY stored anywhere in GitHub. When a workflow runs, GitHub presents a signed JWT to AWS STS. AWS verifies the token against the registered OIDC provider and issues temporary credentials scoped to the portfolio-platform-dev-github-actions IAM role.

The role's trust policy uses a StringLike condition on the token.actions.githubusercontent.com:sub claim, locking it to repo:omaratabany/portfolio-aws-platform:*. A fork of the repo cannot assume this role. Credentials are valid only for the duration of the job, typically under 2 minutes. The deployment workflow only triggers on pushes to main that change files under functions/, avoiding unnecessary deploys on documentation or Terraform-only changes.

OIDC FLOW
git push → main GitHub Actions JWT to AWS STS Temp credentials (role) zip + deploy Lambda

Credentials expire when the job ends. No rotation required. No secret sprawl.

CI also runs a Checkov scan against the Terraform on every PR, in soft_fail mode: findings are visible in the job log, but don't block merges. The first run against already-deployed infrastructure — not a fresh init — came back 130 passed, 45 failed across 25 checks. Hard-enforcing from that baseline would have turned every future PR red over findings nobody had triaged yet, so the real work was triage, not a uniform fix-everything pass:

TRIAGE EXAMPLE FINDING WHY
Fixed SSM SecureString encryption; X-Ray tracing; missing log-group retention (found by inspection, not Checkov) Real, $0, low effort. The log-group gap was the notable one: Lambda auto-creates a log group on first invocation with no expiration — unbounded cost accruing silently until someone checks.
Deferred Customer-managed KMS keys (4 checks); Lambda reserved concurrency; DynamoDB point-in-time recovery Each is a real, ongoing charge against a $0 ceiling — not a rounding error. Reserved concurrency was actually attempted and reverted: this account's Lambda concurrency limit in eu-central-1 is only 10 (not AWS's usual 1000), and is already fully unreserved.
Accepted Public S3 bucket / public API Gateway routes on the status site Checkov is correctly describing what the bucket does — a public status page is the entire point — not catching a mistake.

An automated reviewer (Codex) also caught a real gap on this same pipeline before merge: the workflow's GITHUB_TOKEN had the default scope, which can include write access depending on repo settings, even though neither CI job ever writes back to the repository. Scoped to contents: read with persist-credentials: false on every checkout — a step that runs PR-supplied code with a live, writable token sitting in local git config is exactly the shape of a supply-chain risk, not a hypothetical one.

Alarms That Actually Catch Failures

CloudWatch alarms cover both status-monitor Lambdas: Errors, Duration, and Throttles each, plus an API Gateway 5xx alarm — eight alarms total, still well inside the free 10-alarm allowance. Three of those didn't exist in the first version of this work; an automated reviewer caught all three as real gaps before merge, not stylistic nitpicks:

GAP FOUND WHY IT MATTERED FIX
Errors alarm blind to app-level 500s api.py catches every exception and returns a well-formed 500 response — a successful Lambda invocation from AWS's point of view, so AWS/Lambda Errors never increments even when something underneath is actually failing. Added a separate API Gateway 5xx alarm, watching the response actually sent to the caller instead.
No Throttles alarm This account's Lambda concurrency limit is only 10 and fully unreserved — a burst of concurrent invocations is a documented, real risk, not a hypothetical edge case. Added Throttles alarms alongside each function's Errors alarm.
Duration alarm used Average A 5-minute average lets one genuinely slow invocation get diluted by several fast ones and never cross the threshold. Switched both Duration alarms to Maximum.

The stretch goal for this phase: bridge these AWS metrics into the homelab's self-hosted Grafana, so cloud and on-prem observability sit side by side. The real constraint was structural, not technical — there's no AWS compute for an on-prem Grafana instance to assume an IAM role from, so the usual "role instead of a static key" pattern used everywhere else in this project isn't available here. Built a scoped IAM user instead (read-only: cloudwatch:GetMetricData/ListMetrics/DescribeAlarms and the two non-CloudWatch permissions Grafana's plugin documents needing), with the access key generated directly via the CLI rather than as Terraform state — so the one long-lived credential this project creates never sits in a file under version-control review. Both homelab Grafana instances (a Docker deployment and a separate Kubernetes/kube-prometheus-stack deployment) now query CloudWatch as a data source.

Hit the Real Endpoint

The form below sends a real POST request through this Cloudflare Worker (as a proxy) to the AWS API Gateway endpoint in eu-central-1. Lambda processes it, writes a JSON event to S3, and returns the event ID and S3 key. You're invoking a live serverless pipeline.

LIVE · portfolio-platform-dev-ingest · eu-central-1
POST /ingest · 10 rps throttle

Enter a message and send it. A real AWS Lambda function will receive it, add a UUID and timestamp, write it to S3, and return the storage key.

// Rate limited to 10 req/s · payload capped at 200 chars · proxied via Cloudflare Worker to avoid CORS