Introduction

Pyx is a service that provisions and runs an on-the-fly image transformation pipeline on AWS. You upload originals to S3, request variants through a CDN URL with query parameters (width, height, format), and the system either serves a previously generated file or creates one, stores it for next time, and returns it to the client.

The backend is organized around three concerns: provisioning (S3 buckets, Lambda, CloudFront), transformation (a Lambda worker behind API Gateway), and packaging (a Docker-based build that bundles Sharp for the Lambda runtime).

The package description sums it up: on the fly image transformations—resize and format conversion driven by the URL, with caching so repeat requests stay fast and cheap.

A Next.js frontend (pyx.geekyjunk.in) sits on top of that pipeline: upload originals, tune transform parameters in the UI, and copy a CloudFront-ready URL you can drop into any app.

Why it is needed?

Modern apps serve images across many contexts: thumbnails on listing pages, hero banners on desktop, WebP or AVIF for bandwidth-conscious clients, and legacy JPEG fallbacks. Storing every variant upfront does not scale. Generating everything in the app server on each request wastes CPU and adds latency.

Pyx addresses that gap with a generate-once, serve-many model:

1. Originals live in one place — a dedicated S3 bucket holds source files only.
2. Variants are created on demand — the first request for a given size/format combination triggers Lambda + Sharp; the result is written to a second S3 bucket under a deterministic key (for example, photo__w-300xauto__f-webp.webp).
3. The CDN prefers cache hits — CloudFront is configured with an origin group: it tries the transformed bucket first, and only fails over to the transformation API when the object is missing or the origin errors (403, 404, 5xx).

That pattern gives you:

• No pre-processing pipeline for every possible size at upload time.
• Edge caching after the first transform, so hot images are served from S3/CloudFront, not recomputed.
• URL-driven transforms — clients express intent with query strings, which CloudFront forwards to the API origin.

Architecture

Pyx wires together storage, compute, and CDN layers.

  Client[Client] --> CF[CloudFront]
  CF -->|Primary| S3T[Transformed S3 bucket]
  CF -->|Failover on 403/404/5xx| API[API Gateway + Lambda]
  API --> S3O[Original S3 bucket]
  API --> Sharp[Sharp transform]
  Sharp --> S3T

Request path

1. A client requests an image path (for example /images/hero.jpg?width=800&format=webp) through the CloudFront distribution.
2. Default cache behavior targets an origin group, with query strings forwarded so width, height, and format reach the transformation API when needed.
3. CloudFront checks the transformed S3 origin first. If the object exists, it is served from cache or S3.
4. On miss or eligible error codes, CloudFront fails over to the transformation API—Lambda behind API Gateway.
5. Lambda loads the original from the original bucket, runs Sharp (resize, toFormat), writes the output to the transformed bucket, and returns the image to the client.

Transformed object keys encode the variant so repeat requests resolve to the same S3 key:

{base}__w-{width}x{height}__f-{format}.{format}

Provisioning

A bootstrap script performs idempotent-ish setup:

Step	AWS service	What happens
1	S3	Creates original and transformed image buckets.
2	Lambda	Creates or updates the function from a deployment zip built for the Linux Lambda runtime.
3	CloudFront	Creates a distribution (or reuses one with matching origins) with HTTPS, compression, TTLs, and the origin group described above.

Lambda runtime

The handler:

• Parses query parameters (format, width, height) from API Gateway events (including rawQueryString fallback).
• Resolves the object key from path parameters or raw path.
• Fetches the source image from S3, pipes it through Sharp, optionally resizes and converts format, persists to the transformed bucket, and responds with the correct Content-Type.

Supported output formats include JPEG, PNG, WebP, AVIF, and GIF, with sensible content-type mapping.

Build and deploy toolchain

Pyx relies on a few tools outside AWS itself:

Tool	Role in Pyx
Node.js 18	Runtime for the provisioner and Lambda.
Sharp	High-performance image resize and format conversion (native bindings; must match Lambda’s Linux environment).
Docker	Builds a Linux-compatible Lambda bundle, including Sharp, then extracts the artifact for upload.
archiver	Zips the bundle into the deployment artifact the provisioner uploads.
AWS SDK for JavaScript v3	S3, Lambda, and CloudFront clients for provisioning; S3 read/write inside Lambda.

The flow is: build the Docker image → extract the bundle → run the provisioner to create or update Lambda and related infrastructure.

CloudFront design choices

• Origin group failover — transformed S3 first, API second; status codes 403, 404, 500, 502, 503, 504 trigger failover.
• Query string forwarding — required so cache keys and Lambda see width, height, format.
• HTTPS-only to the API origin, with redirect-to-https for viewers.
• TTL — default 24 hours, max one year; transformed assets are treated as cacheable content.

Frontend usage and integration

The web app is a thin Next.js shell around the AWS pipeline. It does not run Sharp or Lambda itself—it uploads originals to the original S3 bucket and hands you a CloudFront URL with transform query parameters. The first request through that URL triggers the same generate-once path described above.

Live demo: pyx.geekyjunk.in

End-to-end flow

  participant User
  participant UI as Next.js UI
  participant Upload as POST /api/upload
  participant S3O as Original S3 bucket
  participant Prefetch as POST /api/prefetch-asset
  participant CF as CloudFront
  participant Lambda as Lambda + Sharp
  participant S3T as Transformed S3 bucket

  User->>UI: Select image + params, click Optimize
  UI->>Upload: multipart file
  Upload->>S3O: PutObject (timestamp-key)
  Upload-->>UI: { key }
  UI->>UI: Build CDN URL (base + key + query params)
  UI->>Prefetch: { url: assetUrl }
  Prefetch->>CF: GET asset URL (server-side)
  CF->>S3T: cache miss?
  CF->>Lambda: failover transform
  Lambda->>S3O: read original
  Lambda->>S3T: write variant
  Lambda-->>CF: image bytes
  Prefetch-->>UI: ok
  UI-->>User: display output URL

What the UI does

The main page is a client component with a small, focused workflow:

1. Pick an image — file input with local preview via URL.createObjectURL.
2. Set transform options — optional width, height, quality, and format (webp, avif, jpeg, png). Values are sanitized to digits where appropriate.
3. Preview the query string — a live preview shows exactly what will be appended to the CDN URL.
4. Optimize — uploads the file, builds the final asset URL, warms it via the prefetch API, and shows the URL to copy.

When the user clicks Optimize, the app uploads the file, constructs a CDN URL from the returned object key and selected query parameters, then calls the prefetch endpoint so the first transform can run server-side:

const uploaded = await uploadToS3(selectedFile);
const assetUrl = `${cdnBase}/${uploaded.key}${optimizedQuery ? `?${optimizedQuery}` : ""}`;

await fetch("/api/prefetch-asset", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ url: assetUrl }),
}).catch(() => {});

setOutputUrl(assetUrl);

That URL is what you use in <img src="…">, Open Graph tags, or any CMS—CloudFront and Lambda handle the rest on first hit.

API routes (BFF layer)

The frontend keeps AWS credentials off the browser by exposing two Route Handlers:

POST /api/upload

• Accepts multipart/form-data with a file field.
• Writes the object to the original S3 bucket using the AWS SDK.
• Returns { ok: true, key } where key is {timestamp}-{filename}.

POST /api/prefetch-asset

• Accepts JSON { url } and performs a server-side GET to the CloudFront asset URL.
• Exists because a browser fetch() to a cross-origin CDN often needs CORS headers on the distribution; the Next.js server has no such restriction.
• Warms the pipeline: the first GET can miss the transformed bucket, fail over to Lambda, generate the variant, and populate S3—so the user’s copied URL is more likely to be a cache hit immediately after.

Route	Purpose
/api/upload	Write original to S3 from the app server
/api/prefetch-asset	Trigger first transform / cache population without browser CORS

Using Pyx URLs in your own app

After uploading through the demo UI (or your own backend against the same original bucket), you reference images like any other CDN asset:

<img
  src="https://your-distribution.cloudfront.net/1716123456789-hero.jpg?width=800&height=600&format=webp"
  alt="Hero"
  loading="lazy"
/>

Query parameters map to what Lambda reads (width, height, format). The UI also exposes quality in the query string for forward compatibility if your transform handler supports it.

Integration checklist:

1. Provision AWS (buckets, Lambda, CloudFront).
2. Deploy the Next.js app pointed at your original bucket and CDN distribution.
3. Upload originals (UI or direct S3).
4. Use {cdn-base}/{key}?width=…&format=… everywhere you need a variant.
5. Let the first request pay transform cost; subsequent requests hit transformed S3 and the edge cache.

Frontend tech stack

Layer	Choice
Framework	Next.js 16 (App Router)
UI	React 19, Tailwind CSS 4
Upload	AWS SDK v3 S3 client in Route Handler
Deploy	pyx.geekyjunk.in

The frontend is intentionally minimal: one page, two API routes, no client-side AWS SDK. That keeps secrets on the server and makes the app easy to reason about as a control panel for the infrastructure behind it.

Ending note

Pyx is intentionally minimal: a provisioner, one Lambda handler, a Docker-based build for Sharp, and a CloudFront topology that treats transformation as a cache miss handler, not the steady-state path. That keeps the system understandable while still delivering production-shaped behavior—S3 for durability, Lambda for compute, CloudFront for scale.

The Next.js app completes the story for humans and integrators: upload once, copy a parameterized CDN URL, and let the edge do the rest.

To adopt it in your own account, run the Docker bundle steps, execute the bootstrap script to stand up buckets, function code, and distribution, then deploy the frontend against that infrastructure. Upload originals and let the first CDN miss pay the transform cost; every hit afterward should prefer the transformed bucket and the edge cache.

If you extend Pyx, good next steps include Lambda Layers (smaller deploy artifacts), smarter CloudFront URL canonicalization, and more reliable idempotent provisioning—without changing the core idea: one URL, many images, generated once and served from the edge.