Two Hundred Million Redirects

Our link shortener does about 200 million redirects a day. Every redirect fires a click event and we need to serve two consumers from that stream: a real-time dashboard that shows per-link clicks within the last hour, and a nightly batch aggregate that powers the analytics API for date-range queries. Traffic is very spiky and some links go viral. Design the pipeline.

Business Requirements

Link creators check their dashboard right after sharing and need clicks visible within roughly a minute; perceived lag is a top support complaint.
The analytics API serves single-link lookups and date-range scans for an account; today the flat layout makes both slow.

+ Source

+ Transform

+ Storage

+ Quality

+ Consumer

+ Queue

Bronze

Silver

Gold

Custom

Pipeline Architecture

Sketch the architecture.

Click or drag a node from the toolbar above. Right-click the canvas for the full menu.

Drag from a node's right port to another node's left port to wire data flow.

Active Now|Sr. (L5)||6.0k Attempts|2.3k Solves|