Building the System Nobody Notices

By Jared Leonard · Senior Engineer, AWS · July 27, 2026

Part 8 of 8: Series index

The posts before this one each pulled a single thread: the one thread you must not block, the engines you don't control, the patterns that leak when ported, the cost of the DOM, workers, memory. This post weaves them together by building something. On paper, with you watching, we are going to design the kind of system that needs every lesson in this series at once, and I can walk you through the walls before you hit them, because this is the problem shape I have spent years of my career inside.

Here is the shape. A user has data scattered across dozens of independent backend services, and they want one screen that shows all of it. Admin panels over microservices have this shape. Monitoring and analytics dashboards have it. Any product whose data outgrew a single backend has it. The services are separate by design, there is no single endpoint that returns "everything you have," and the unit of retrieval is one call per data type per service. Tens of types times dozens of services lands you, very quickly, at roughly a thousand calls to render a single screen.

The first decision: where does aggregation live

The reflexive answer is a backend: stand up an aggregation tier, a global cache, a team to operate it, a budget line, and a new set of failure modes. Sometimes that's right. But look at what's already sitting in front of you: a machine on the user's desk with a modern JS engine, a local database, a worker pool, and network access, whose costs scale to zero when nobody is looking at the screen. Serve the assets off a CDN, hand the tab short-lived credentials, and let the calls go out directly from the browser. The tab becomes the aggregation layer, and when the global API you need doesn't exist, building it at the edge is sometimes the cheapest correct answer.

That decision is the whole reason this series exists. The moment a browser is fanning out a thousand calls, reconciling the responses, caching them, and keeping a UI alive through all of it, you are not writing a web page. You are operating a distributed-systems node.

Treat it like an app, not a page

A page fetches some data and renders it. An app orchestrates: it schedules work, manages concurrency, caches across sessions, reconciles state, and degrades when a dependency is slow. The distinction changes which problems you can even see. Think "page," and a thousand API calls is a data-fetching problem: fire them, render what comes back. Think "app," and it's a scheduling problem, which means a work queue, a concurrency budget, and a strategy for getting results onto the screen without setting the screen on fire.

The page version is where everyone starts, so let's start there too.

The wall you will hit

The naive build works, and that's what makes it dangerous. Fire all the calls; as each response lands, push it into the store; the UI re-renders; the user watches the screen fill in. At a few hundred calls this is completely fine, and it will stay fine long enough for you to ship it, get promoted off it, and hand it to someone else.

Then the product grows. More data types, more services, the call count climbs toward a thousand, and the bottleneck that emerges is you rather than the network. Every response dispatches a store update, every update triggers reconciliation, and a thousand updates arriving in a burst become a re-render storm on the one thread the browser gives you. The network keeps up fine; the main thread drowns in your chatter about the data rather than the data itself. This is the single-threaded covenant and the true cost of the DOM collecting their debt at the same time, and the fix is about changing how you talk to the browser, not about fetching faster.

Getting the work off the main thread

First move: the orchestration leaves the main thread entirely. The fan-out, the response handling, the cache writes, the reconciliation all belong in Web Workers, so the thread keeping the UI alive is no longer the thread doing the lifting.

One worker isn't enough, because one worker is now the bottleneck for a thousand tasks. The shape that works is a worker pool fed by a task queue: each call (one data type from one service) is a task, tasks go on the queue, and a small pool drains it with a bounded concurrency. The bound is the point, more than the pool. It exists so the rate of incoming results is something you control rather than something the network dictates. A queue with a concurrency cap is the edge's version of backpressure.

Do not expect one bound to fit everywhere. This is the engines post coming due: the concurrency that's comfortable in one browser family will misbehave in another, and you don't get to pick the runtime your users bring. Plan for the safe setting to be discovered per engine, and make it a config value rather than a constant, because you will be changing it.

How the workers talk back

Moving work into workers solves half the problem and mints the other half: the results now live in a different context from the UI, and the bridge between them becomes the new place to flood the main thread. A worker that postMessages once per response has just relocated the storm.

The fix is to stop treating worker-to-page communication as a pile of point-to-point pipes and treat it as a bus, with a BroadcastChannel shared between the workers and the page. That buys you a few things:

• One shared bus instead of N pipes. A pool of workers all reporting into a single channel the page subscribes to once beats a separate handler per worker.
• Coalescing and batching. Results accumulate and get announced in batches, so store updates (and therefore re-renders) happen on your cadence, not the network's. Let the work rate be set by what the UI can absorb, not by how fast events arrive.
• Decoupling the writer from the reader. For the heavy paths, workers write results into IndexedDB and broadcast only a small "this changed" signal. The page reads on its own schedule, and the large payload never rides the main-thread message path at all.

None of these are exotic primitives. Web Workers, BroadcastChannel, and IndexedDB are sitting in every browser. What's non-obvious is treating them as the parts of a distributed system (a worker pool as a thread pool, a channel as a message bus, a local database as a cache tier) instead of as page-scripting conveniences.

The part the user actually feels

The last piece is what makes it feel instant, and it's pure cache-aside. You don't need the full dataset to draw something useful: rendering the grid of "what you have and where" takes a handful of fields per resource. So the local database gets read first, the skeleton paints in well under the time the network needs to answer, and the workers fan out behind it to fill in detail and freshness. On a second visit the screen is there before the user has finished deciding what they came to look at.

Caching at the edge buys you that and bills you for staleness in return. The cost is a reconciliation pass: when fresh data comes back, anything in the cache that the latest scan didn't return has to be swept out, or you will happily show people things they deleted last week. A timestamp per record and a periodic sweep is the unglamorous machinery that keeps "instant" from quietly becoming "wrong." It's the same lifecycle discipline from the patterns post: a cache without an eviction story is a bug with a latency benefit.

Why nobody will notice

Here is the strange economics of doing all of this well: the payoff is invisible. Users never praise latency. When a screen like this works, the reaction is about what it showed them, the forgotten thing nobody remembered creating, the item that should have been switched off months ago, never about how many milliseconds it took to assemble the view. And every bit of that value depends on the invisible part, because if the screen took half a minute and froze the tab, they'd have bounced before the value appeared. Invisible infrastructure holding up a visible result is the entire job, and you only hear about latency once you've lost it.

The series, in one system

Every post in this series is somewhere in this design. The single thread you must not block runs through all of it. The engine differences you can't assume away set your concurrency caps. The server patterns needed a lifecycle before they were safe, and the DOM cost punishes you for talking too much. The workers are the only real escape hatch, and they arrive with messaging economics of their own. The memory is something you negotiate for rather than own.

If you keep one artifact from the series, make it the translation. The left column is the toolkit you already run on servers; the right column is what it answers to in a browser:

The vocabulary transfers; the implementations adapt. Half those rows appeared in this one design.

You rarely set out to build a distributed system in a tab; you set out to build a screen, hit the wall every edge node eventually hits, and discover that the way out is to stop treating it like a page and start treating it like the app it has quietly become. That's the whole thesis. The browser is the node closest to your user, doing harder work than anyone credits it for, and when you engineer it as such, the best outcome you can hope for is that nobody ever notices how hard it was.

Series complete. Back to the index →