API Gateway and Composition Lab
A shop's product page needs data from up to eight services: product, price, stock, reviews, recommendations, delivery estimate, seller and promotions. Decide who calls them, in what order and with what protection, then load the page 2,000 times and see how fast the typical page is, how slow the slowest are, and how many come back complete.
Background: API Gateway, API Composition, Backend for Frontend, Timeout.
Press Play to load the page.
Pages slower than 1.0 s, with the same settings and 1 to 8 services
Run these in order
Let the phone call every service
The product page needs six services, and the app calls each one itself, one after another. Every service is a different host, so each call opens a new connection over a mobile network with a 150 ms round trip: two round trips to connect, one for the request. The median page takes over 3 seconds. Switch to “At the same time”: about 0.6 s, but the phone still pays three round trips and knows the address of every service.
Put a gateway in front
The phone makes one call to a gateway, which calls the six services inside the data centre at the same time. The median page drops to about 0.3 s. Now look at the p99: about 2 seconds. A few pages in a hundred hit one service having a pause, and the page is only as fast as its slowest call. Switch to “One after another”: the gateway then waits for the sum of the calls, and the median doubles.
Add services and watch the tail
Compose the page from all eight services. Each optional service pauses on about 1 call in 80, yet over 6% of pages now take more than a second. The chart under the results runs the same settings with one to eight calls: with one call, 3 pages in 1,000 are slow and the p99 is under 0.3 s. Every call you add is one more chance to wait for a pause.
Slow down Reviews
The Reviews service starts taking three seconds longer than usual. With no timeout the gateway waits for it, so every page takes over 3 seconds, although the product, price and stock arrived in 50 ms. Now set a 500 ms timeout: the gateway stops waiting, but because the page needs every section, every page fails.
Render a partial page
Keep the 500 ms timeout and let the page render without an optional section that failed or timed out. Pages are back to under 0.7 s, showing the product, price and stock with “Reviews are not available right now”. Nearly no page is complete, and a handful still fail: those hit a pause in a required service, and a timeout on Price cannot be hidden.
Cache what changes slowly
The gateway keeps product details, reviews and seller data in a cache, and nine in ten requests are served from it. Most pages are complete again even while Reviews is slow, and the median is back to 0.3 s. Now make Price fail instead: the cache and partial pages cannot help, because a product page without a price is no page at all. Required services need their own protection.
This lab simplifies reality on purpose. Every load is independent, with no traffic peaks and no queueing in the services or the gateway, each service's pauses are random rather than correlated, and the mobile network has a fixed round trip with no packet loss. It leaves out retries and hedged requests, circuit breakers, per-request time budgets, HTTP/2 connection reuse across hosts, and what the gateway itself costs to run and scale. Real composers also stream sections to the page as they arrive rather than waiting for all of them.