What is the Health Check API pattern?
A service exposes HTTP endpoints that the orchestrator or load balancer polls. A liveness check answers 'is this process stuck?' and failing it gets the instance restarted. A readiness check answers 'can this instance serve requests right now?' and failing it takes the instance out of rotation without restarting it.
What problem does the Health Check API pattern solve?
A process can be running but useless: deadlocked, still warming up, or cut off from something it needs. Without health checks the platform keeps sending it traffic. With checks that are too aggressive, the platform restarts healthy instances and turns a small problem into an outage.
How do you implement the Health Check API pattern?
- Keep liveness simple: check that the process can respond and its main loop is not stuck.
- Make readiness check what this instance needs to serve, such as config loaded and caches warm.
- Do not fail liveness because a shared dependency is down, or every instance restarts at once.
- Keep checks cheap and fast, and cache any dependency checks for a few seconds.
- Set probe intervals, thresholds and startup grace periods to match how the service actually starts.
When should you use it?
- Any service behind a load balancer or run by an orchestrator.
When should you avoid it?
- Never skip it. Avoid the deep checks described in the mistakes below instead.
What do you gain?
- Traffic only goes to instances that can serve it
- Stuck instances are replaced automatically
- Safer rolling deploys
What does it cost?
- Bad checks cause restart loops
- Deep checks add load to shared dependencies
- Needs tuning per service
What are common mistakes with the Health Check API pattern?
- Liveness checks that call the database, so a database blip restarts the whole fleet.
- A readiness check that always returns 200, so new instances get traffic before they are ready.
- Expensive checks that, multiplied by every instance and every few seconds, become real load.
Health Check API in practice
Edgewise CDN
A service with deadlocked worker threads still answered the load balancer's TCP check, so traffic kept flowing to an instance that could not serve it.
The service now exposes separate liveness and readiness endpoints. Liveness checks that the worker loop is responsive. Readiness checks that the instance has loaded what it needs to serve. Stuck instances are restarted, and new ones get no traffic until they are ready.
Example decision record
- Context
- New instances receive traffic before their caches are warm, causing bursts of errors on every deploy.
- Decision
- Expose separate liveness and readiness endpoints from every service.
- Consequences
- The platform sends no traffic until readiness passes.
- Probes stay cheap so they do not overload shared dependencies during a mass restart.