SLO Burn Rate Alerts Generator

A service can still be inside its reliability target while a current incident is consuming the remaining allowance fast enough to put that target at risk. Burn-rate alerting measures that speed instead of waiting for the full compliance period to fail.

Service level indicator (SLI)

The measured ratio of good events to total events, such as successful requests or requests below a latency limit.

Service level objective (SLO)

The target for that indicator over a stated compliance period.

Service level agreement (SLA)

A commitment with contractual or business consequences. An SLO may inform an SLA but is not automatically the same thing.

Error budget

The allowed bad-event ratio implied by the SLO. A 99.9% objective leaves a 0.1% budget.

Burn rate

The speed of budget consumption relative to a rate of 1, which would exhaust the budget exactly at the end of the compliance period.

A high burn multiple calls for a quick response because the budget could disappear in hours. A lower but sustained multiple may justify a ticket instead of a page. Several tiers let the notification urgency follow the time available to act.

Each tier uses a long window and a short window at the same threshold. The long window asks whether enough budget has been consumed to matter; the short window asks whether the damaging rate is still present. Requiring both reduces alerts that remain active only because an old incident is still visible in a long average.

Traffic volume sets an important limit. On a low-traffic service, one failed request can produce an extreme short-window error ratio even when the sample is too small for a useful page. Query correctness, denominator stability, missing-data behavior, and alert routing therefore need operational testing alongside the budget math.

Generated rules are a reviewable starting point. They do not prove that the SLI represents user experience, that Prometheus returns the intended series, or that Alertmanager will deliver the notification to the correct responder.

How to Use This Tool:

Use the same objective, compliance period, and bad-event definition as the production SLO policy.

1. Enter Service name, SLO label, the target percentage, and the compliance period. Confirm that the target describes successful events rather than the error ratio.
2. Choose the YAML Rule format used by the destination and set the group Evaluation interval.
3. Paste an Error-ratio query template that returns bad events divided by total events as a decimal ratio. Keep the literal {window} placeholder; optional {service} and {slo} placeholders are filled from the labels.
4. Edit Burn-rate tiers as key, severity, long window, short window, burn multiple, and for duration. The short window must be strictly shorter than the long window.
5. Add routing labels and optional runbook or dashboard links only when they match the notification policy. Enable Keep firing after recovery only for a Prometheus version and alert lifecycle that support it.
6. Inspect Budget ledger and Routing plan, then read every generated expression. Validate the saved rule with promtool check rules or the destination admission path before any reload or deployment.

Interpreting Results:

Error threshold is a decimal bad-event ratio. A displayed value of 0.0144 means 1.44% bad events, not 0.0144%. Each generated expression uses the strict operator >, so equality does not pass the threshold.

Both the long-window and short-window queries must be greater than the same threshold. The condition must then remain active for the tier’s for duration before the alert fires. When included, keep_firing_for extends the firing state after the expression clears; it does not change the threshold.

Budget share estimates the percentage of the full compliance-period budget consumed at the tier’s burn rate over its long window. Exhaustion is the time a constant burn rate would take to consume the entire budget. These are policy projections, not forecasts of the current incident.

A clean ledger does not verify the PromQL denominator, label set, live sample count, route, or notification. Test the query over representative traffic, validate the YAML, and exercise the notification path before treating the rule as production-ready.

Technical Details:

The computation converts an SLO success percentage into an allowed error ratio, scales that budget by each burn multiple, and evaluates two lookback queries against the resulting threshold. Compliance duration affects the budget-share and exhaustion projections but not the threshold ratio itself.

Formula Core: Budget and threshold math

The following equations use unrounded values, with final model outputs rounded to at most 12 decimal places.

For a 99.9% SLO over 30 days, B is 0.001 and H is 720 hours. A 14.4× tier produces θ = 0.0144, consumes 2% of the budget over a 1-hour long window, and has a 50-hour exhaustion horizon.

Rule Core: Multi-window tier logic

The built-in tier set follows the common 12:1 long-to-short window guideline and adds both urgent and scheduled-response paths. These values are starting points, not universal policy.

Each tier becomes (long query > θ) and (short query > θ). The generator warns when θ is at least 1, because that requires a 100% or greater error-ratio threshold, and when the long-to-short ratio is below 6:1 or above 24:1. A tier burn multiple must be greater than 0 and no more than 100,000; up to 12 tiers are accepted.

Durations use positive Prometheus duration syntax and may combine supported units from milliseconds through years. The rule group interval controls evaluation cadence. A tier’s for duration controls how long its expression must remain active, and optional keep_firing_for controls how long the firing state remains after recovery.

The selected wrapper changes only the surrounding artifact: a plain Prometheus rule file, a Prometheus Operator PrometheusRule resource, or a Kubernetes ConfigMap carrying a rule file. The tier expressions, labels, annotations, and budget math remain the same.

Accuracy Notes:

The generated YAML is deterministic from the supplied policy. Operational correctness still depends on systems and data that are not inspected here.

• Confirm that the PromQL expression returns a decimal bad-event ratio with the intended denominator and label scope.
• Test sparse and missing-data periods; low traffic can make a single failure dominate a short window.
• Run promtool check rules, query representative windows, and verify pending, firing, recovery, and routing behavior.
• Check that runbook and dashboard links are reachable by responders and that page-level alerts are immediately actionable.

References:

• Alerting on SLOs, Google SRE Workbook.
• Alerting rules, Prometheus documentation.
• Defining and validating rule files, Prometheus documentation.
• How to load a Prometheus rule file, Simplified Guide.