What is the Timeout pattern?
Every remote call gets an explicit time limit, both for connecting and for the whole request. When the limit passes, the caller gives up, frees its resources and handles the failure. With deadline propagation, the remaining time travels with the request so downstream services stop work nobody is waiting for.
What problem does the Timeout pattern solve?
The default for many clients is no timeout or a very long one. A dependency that hangs then holds a thread, connection or memory on every caller, and callers of those callers, until the whole chain runs out. A slow dependency is more dangerous than a dead one, and timeouts are what turn slow into failed.
How do you implement the Timeout pattern?
- Set connect and request timeouts on every client: HTTP, gRPC, database and broker.
- Base the value on the dependency's measured latency (for example a little above p99), not on a round number.
- Give the whole user request a budget and pass the remaining deadline downstream.
- Make each layer's timeout shorter than the one above it, and count retries inside the budget.
- Decide what happens on timeout: a fallback, an error, or a retry if the call is idempotent.
When should you use it?
- Always, for every call that crosses a network.
When should you avoid it?
- Never skip it. For work that is legitimately slow, change the interaction to asynchronous instead of raising the timeout.
What do you gain?
- Bounds how long failures hold resources
- Makes slow dependencies fail visibly
- Cheap to add
What does it cost?
- Too short causes failures under normal load
- A timed-out request may still have succeeded on the server
- Needs tuning per dependency
What are common mistakes with the Timeout pattern?
- Relying on library defaults, which are often infinite or minutes long.
- Inner timeouts longer than outer ones, so the caller has given up but the work continues.
- Retrying a non-idempotent call after a timeout, when the first attempt may have worked.
Timeout in practice
In a deep tree of RPCs, a backend can keep working on a request long after the frontend that asked for it has given up, wasting capacity exactly when the system is overloaded.
Google's SRE book recommends deadline propagation: set one absolute deadline high in the stack and give each downstream RPC only the time that remains, so servers can drop work nobody is waiting for. It pairs this with cancellation propagation, so a failure or timeout stops sibling RPCs too.
Source: Site Reliability Engineering, ch. 22: Addressing Cascading Failures (Google)
Example decision record
- Context
- Latency spikes in downstream services cause the gateway to run out of worker threads.
- Decision
- Set explicit connect and request timeouts on every outbound call, and propagate a deadline with each request.
- Consequences
- No request can wait forever on an unresponsive dependency.
- Callers need a defined behaviour when a timeout fires.