Load Balancer Failover Time Calculator
Estimate user-visible load balancer failover time across health checks, routing changes, DNS cache exposure, and an RTO target.| Milestone | Window | Planning meaning | Copy |
|---|---|---|---|
| {{ row.label }} | {{ row.value }} | {{ row.note }} |
| Check | State | Evidence | Next action | Copy |
|---|---|---|---|---|
| {{ row.label }} | {{ row.state }} | {{ row.evidence }} | {{ row.action }} |
Introduction
A service can stop answering in an instant, yet traffic usually takes longer to reach a healthy replacement. The delay is assembled from several clocks: a health probe must notice the failure, enough consecutive checks must fail, routing must change, and some clients may continue using cached DNS answers or existing connections.
Health checks create the first timing boundary. The check interval says how often a backend is tested, the timeout limits how long each failed response may take, and the unhealthy threshold says how many failures are needed before traffic is removed. Shorter intervals and smaller thresholds reduce detection time, but they also make a service more sensitive to brief latency, packet loss, deploy pauses, or a health endpoint that does not represent real user traffic.
The moment a failure begins within the probe cycle matters too. A fault just before the next check is detected sooner than one just after a check. Planning for the worst case adds a full interval before the failed checks begin. An average-case estimate adds half an interval, while a trace that starts at the first failed probe adds no alignment wait.
| Traffic path | Timing that matters | What may remain behind |
|---|---|---|
| New traffic, same endpoint | Health-check detection plus routing update | Existing sessions may still use the old backend |
| New traffic, changed DNS answer | Detection, routing update, authoritative change, and recursive TTL | Resolvers and applications may retain the old answer |
| Return to service | Successful checks plus routing update | A recovered backend may wait for the healthy threshold |
A recovery time objective (RTO) is a service commitment, not a health-check setting. Comparing a modeled user-visible delay with the RTO can expose an obvious mismatch, especially when DNS time to live (TTL) dominates the path. It cannot prove that standby capacity, application state, database recovery, retries, sticky sessions, or the health endpoint itself will behave correctly during an incident.
How to Use This Tool:
Build the scenario from the client-visible path, then replace preset values with measured or documented production settings.
- Choose a Provider profile as a starting point, or select Custom when the deployed settings are known.
- Select Failover path. Include DNS only when failover can change the A, AAAA, or CNAME answer seen by clients.
- Enter the probe interval, timeout, unhealthy and healthy thresholds, reroute delay, and any connection drain. Use the platform's actual scheduling behavior when checks do not follow the sequential timing assumed here.
- Set the DNS TTL and authoritative propagation delay for DNS-based paths, then choose the failure-timing assumption and add a jitter buffer for orchestration or measurement uncertainty.
- Compare User-visible failover and RTO margin with the target. Review the largest contributor before tightening probes or lowering TTL.
- Test the proposed settings with a controlled failure. Confirm both new-request routing and the behavior of existing sessions.
Interpreting Results:
User-visible failover is the main RTO comparison. New traffic failover excludes DNS cache exposure, while Existing connection cleanup adds the drain window to the new-traffic path. Return to service is a separate estimate for accepting a recovered backend again.
A result inside the target is planning evidence, not a guarantee. Recheck the estimate when probe scheduling, DNS caching, connection reuse, or routing propagation differs from the assumptions, and verify the final timing under realistic load.
Technical Details:
Sequential health checks accumulate timeout windows and the interval gaps between them. Alignment wait is added before the first modeled failure, while jitter represents unmodeled delay around detection and automation. DNS terms are added only when the client endpoint can change.
Formula Core
The equations below separate failure detection, new-traffic routing, DNS exposure, existing-session cleanup, and recovery.
| Symbol | Meaning | Unit |
|---|---|---|
| A | Alignment wait: one interval, half an interval, or zero | seconds |
| I, T | Health-check interval and probe timeout | seconds |
| n, h | Unhealthy and healthy consecutive-check thresholds | checks |
| C, J | Reroute delay and jitter buffer | seconds |
| P, L, G | Authoritative propagation, recursive TTL, and connection drain | seconds |
| D, N, U, E, R | Detection, new-traffic, user-visible, existing-connection, and return-to-service times | seconds |
For backend-only paths, P and L are zero. RTO margin equals target RTO minus U, and utilization is U divided by the target, expressed as a percentage. Calculations retain full precision; display precision changes formatting only.
Rule Core
| Check | Boundary | Status |
|---|---|---|
| RTO utilization | At most 75% | Comfortable |
| Above 75% and at most 100% | Inside RTO | |
| Above 100% and at most 150% | Over RTO | |
| Above 150% | Critical | |
| DNS TTL when DNS is used | At most 120 seconds | Clear |
| Above 120 and at most 300 seconds | Watch | |
| Above 300 seconds | Action |
Probe cadence is marked for action when the interval is below one second, or when a sub-five-second interval uses a one-failure threshold. A timeout longer than the interval, any other sub-five-second interval, or a one-failure threshold receives a watch status. These flags identify aggressive assumptions; they do not declare a provider configuration invalid.
Accuracy Notes:
The model assumes sequential checks and additive delays. Some platforms schedule probes independently, remove targets before a timeout fully elapses, apply distributed consensus, or keep retry and fail-open behavior outside these terms.
- Use a measured health-check timeline when the provider's scheduler differs from the sequential formula.
- Treat TTL as planned cache exposure, not a promise that every resolver or application obeys the same expiry.
- Validate application readiness, state recovery, retry behavior, standby capacity, and session handling separately.
Worked Examples:
Combined backend and DNS path
With a 10-second interval, 5-second timeout, two failed checks, worst-case alignment, and no jitter, detection takes 30 seconds. A 5-second reroute delay brings new-traffic failover to 35 seconds. Adding 30 seconds of authoritative propagation and a 120-second TTL produces a 185-second user-visible estimate, which is 5 seconds over a 180-second RTO. A 60-second drain makes existing-connection cleanup 95 seconds because DNS exposure is not added to that path.
References:
- Health checks for Application Load Balancer target groups, Amazon Web Services.
- Azure Load Balancer health probes, Microsoft Learn.
- Health checks overview, Google Cloud.
- How to configure backend health checks in HAProxy, Simplified Guide.