HA Pair Role Consistency Checker

Introduction

High availability pairs are meant to make one service survive a node failure without forcing users to choose a new address or restart their work. The usual active/passive design has one peer serving traffic and another peer waiting with enough configuration, state, and control-plane contact to take over. That simple story becomes harder to prove during maintenance windows, firewall upgrades, routing changes, database patching, or incident review because the evidence rarely arrives as one clean yes-or-no value.

A useful HA review starts by separating role from readiness. A row that says active is making an ownership claim. A row that says up is making a health claim. A row that says synced, heartbeat up, or preempt on is describing supporting evidence around that claim. These signals can disagree. The active peer can be marked down, the standby peer can be healthy but stale, two peers can both claim the active role, or the virtual IP address can differ between rows that are supposed to describe the same protected service.

Split brain is the most urgent role-consistency failure. It happens when more than one peer acts as if it owns the same protected service. Even when no split-brain symptom appears, a false sense of redundancy can still be costly. A down standby, stale session sync, or broken heartbeat path may leave the active peer serving traffic with no trustworthy failover target. Planned work that would normally be routine can become risky because the pair is no longer in the state the runbook assumes.

Priority rules deserve extra care because they are not universal. VRRP-style systems normally prefer a higher-priority peer, and preemption can let a recovered preferred peer reclaim ownership. Other platforms pin roles, invert scores, suppress failback, or use priority-like values only as hints. The safe question is not only which number is larger, but whether the current owner agrees with the platform convention that applies to that service.

A row-based consistency check can catch drift in the evidence before a failover test or change window. It cannot prove that every dependency will move correctly. Quorum, fencing, route advertisements, storage locks, session replication, application warm-up, and vendor-specific failover rules still need direct product checks.

How to Use This Tool:

Paste the HA evidence you already have, then set the expected ownership and readiness rules for the service group you are reviewing.

1. Paste CSV or tab-delimited rows into HA role rows. A header row is optional. Without a header, the expected order is node, role, priority, VIP, state, sync, heartbeat, and preempt. Lines that start with # are ignored.
2. Include a group, service, or VIP value when one paste contains more than one protected service. Rows with the same group are checked together, and rows without a group fall back to the VIP or a default pair name.
3. Set Expected active-up to the number of peers that should be both active-style and up-style. Ordinary active/passive pairs usually stay at 1.
4. Set Minimum standby-up to the number of standby-style peers that must be up before the group counts as ready for failover. A two-node pair usually needs 1.
5. Choose Priority rule according to the platform convention. Use higher priority should be active for VRRP-like election behavior, lower priority should be active for inverted conventions, or ignore priority when ownership is intentionally pinned outside numeric priority.
6. Set Sync stale after for numeric sync lag. Lag above that value is treated as stale. Text values such as synced, stale, lagging, and out-of-sync are read by meaning.
7. Start with Pair Health Table for the group verdict. Use Node Role Ledger to inspect each peer, Failover Findings for failed checks and next steps, and Priority Role Ladder when numeric priorities need visual comparison.

If Check HA input rows appears, fix missing node names, missing roles, or too few node rows before trusting the results. Parsed rows may still appear, but the redundancy judgment is incomplete until the validation warning is gone.

Interpreting Results:

The most important fields are Active-up, Standby-up, Sync, and Verdict in the Pair Health Table. A clean verdict means the supplied rows match the selected rules that could be checked. A critical verdict means at least one core failover condition failed, such as the active owner count, standby readiness, active state, or heartbeat health.

Avoid false confidence from a clean group count alone. The Pair Health Table summarizes the group, while Node Role Ledger and Failover Findings show which row and which rule created the verdict. Use the row evidence when deciding whether to proceed, pause, or collect a fresher export.

Technical Details:

The checker treats each row as one HA peer observation. The evaluation boundary is the group name when present, then the VIP or service value when the group is blank, and finally a default pair label when neither is supplied. Inside each group, the rules compare normalized role, normalized state, numeric priority when available, sync freshness, heartbeat status, preempt values, duplicate node names, and service identity.

Role normalization keeps common HA vocabulary comparable while preserving the original row text in the ledger. Active-style roles include active, primary, master, and owner. Standby-style roles include standby, passive, secondary, backup, and spare. State is evaluated separately because role claims do not prove readiness. A peer can be active and down, standby and up, standby and stale, or unknown enough to require manual review.

Rule Core:

Sync and heartbeat are intentionally judged differently. Sync freshness warns about stale configuration or session state. It affects confidence in a clean handoff, but it does not always mean the current active peer stopped serving traffic. Heartbeat failure is more severe because the peers may not be exchanging the control messages needed to detect failure and coordinate ownership.

Priority ownership is checked only when it can support a fair conclusion. The group must have exactly one active-up peer, at least one other up peer with numeric priority, and a selected priority convention other than ignore. If the active peer has no numeric priority, the result stays informational. If another up peer outranks the active peer under the selected rule, the result becomes a warning rather than a critical failure because no-preempt designs and manual role pinning can be valid.

The Priority Role Ladder is a chart of numeric priority rows. It is useful for spotting a high-priority standby, a down preferred peer, or an active owner below another up peer. The formal pass/fail decision still comes from the rule checks and the Failover Findings table.

Limitations and Privacy Notes:

The checker reviews consistency in the rows you provide. It does not contact HA devices, query controllers, confirm traffic ownership, or replace vendor failover procedures.

• Pasted HA rows are evaluated in the browser. There is no server lookup or upload step for the inventory data.
• Review the address bar before sharing a filled page because repeatable checks can carry values in the current URL.
• A clean result does not prove quorum, fencing, route convergence, session replication, shared-storage locking, or application readiness.
• Normalize uncommon vendor wording when the ledger marks role, state, sync, or heartbeat values as unknown.

Worked Examples:

FAQ:

Glossary:

Active

The peer currently claiming ownership of the protected service or virtual address.

Standby

The peer expected to be ready to take over when the active peer fails, is maintained, or is moved.

Active-up

A row that is both active-style and up-style, counted against the expected owner count.

Standby-up

A row that is both standby-style and up-style, counted against the required ready peer count.

VIP

A virtual IP address or shared service address that should follow the active role.

Heartbeat

The HA control-path signal used to monitor whether a peer is reachable and participating.

Preempt

A behavior where a preferred peer may reclaim ownership after it becomes eligible again.

Split brain

A dangerous condition where more than one peer acts as if it owns the same protected service.

References:

• RFC 9568: Virtual Router Redundancy Protocol (VRRP) Version 3 for IPv4 and IPv6, RFC Editor, April 2024.
• Introduction to High Availability, Ubuntu Server documentation.
• High Availability Add-On overview, Red Hat Documentation.
• Keepalived for Linux, Keepalived project.