Keepalived VRRP Instance Stuck in FAULT State: How to Diagnose and Fix Transition Failures

If you've ever SSH'd into a server expecting to find your Keepalived instance happily serving as MASTER, only to find it sitting in FAULT state while your VIP is nowhere to be found — you know the particular frustration this article is about. FAULT state is Keepalived's way of telling you "something is seriously wrong and I'm refusing to participate in VRRP," but it's not always obvious what that something is.

Let's work through this systematically. By the end of this article, you'll have a clear diagnostic checklist and the fixes for the most common culprits.

Understanding FAULT State: What It Actually Means

VRRP instances transition to FAULT state when a tracked resource fails — specifically when a script check or interface track fails enough times to bring the instance priority to zero or below. This is distinct from an instance simply losing an election (which would leave it in BACKUP, not FAULT).

The key mental model here: FAULT means "I've deliberately taken myself out of the game." The instance won't send VRRP advertisements, won't hold the VIP, and won't respond to elections until the underlying failure is resolved.

Common paths into FAULT state:

• A track_script is returning a non-zero exit code
• A tracked interface (track_interface) has gone down
• The instance priority was reduced to zero or below due to accumulated weight penalties

Step 1: Confirm the State and Check Logs First

Before you touch anything, confirm what you're dealing with:

# Check current VRRP state
systemctl status keepalived

# Or send a signal to get state dump
kill -USR1 $(pidof keepalived)
cat /tmp/keepalived.data

The real gold is in the logs. On most systemd-based systems:

journalctl -u keepalived -n 100 --no-pager

You're looking for lines like:

VRRP_Script(chk_haproxy) failed
VRRP_Instance(VI_1) Entering FAULT STATE
Tracker weight would bring instance below zero

These tell you why the transition happened. Don't skip this step — it saves you from guessing.

Step 2: Audit Your Track Scripts

Track scripts are the #1 cause of unexpected FAULT states in my experience. A script that works fine at the command line can behave completely differently when run as the keepalived user with a restricted environment.

Here's a typical problematic config:

vrrp_script chk_haproxy {
    script "/usr/bin/killall -0 haproxy"
    interval 2
    weight -50
    fall 3
    rise 2
}

Test the script manually as the keepalived user:

sudo -u keepalived /usr/bin/killall -0 haproxy
echo $?

If you get a non-zero exit code, you've found your problem. Common script issues include:

• PATH problems: The script can't find the binary because PATH isn't set for the keepalived user
• Permission issues: The keepalived user can't read/execute the script or the thing it's checking
• Missing dependencies: The script relies on a tool that isn't installed

Fix PATH issues by using absolute paths in your scripts, or by wrapping them:

#!/bin/bash
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/usr/bin/systemctl is-active --quiet haproxy

Then reference it in your Keepalived config:

vrrp_script chk_haproxy {
    script "/etc/keepalived/scripts/check_haproxy.sh"
    interval 2
    weight -50
    fall 3
    rise 2
    user root
}

Note the user root directive — for scripts that need elevated privileges, this is cleaner than fighting with sudo permissions.

Step 3: Check Your Weight Math

This one bites people more than they expect. VRRP instances have a base priority, and track scripts can apply negative weights when they fail. If the sum of weights from failed scripts would bring your priority to zero or below, the instance enters FAULT.

Let's say you have:

vrrp_instance VI_1 {
    state MASTER
    priority 100
    
    track_script {
        chk_haproxy
        chk_nginx
    }
}

vrrp_script chk_haproxy {
    weight -60
    ...
}

vrrp_script chk_nginx {
    weight -60
    ...
}

If both scripts fail simultaneously, your effective priority becomes 100 - 60 - 60 = -20. That's negative, so FAULT state. The fix is either to increase your base priority or reduce your weight penalties so the math can't go negative:

vrrp_instance VI_1 {
    state MASTER
    priority 150    # Increased to survive both scripts failing
    ...
}

Or use more conservative weights:

vrrp_script chk_haproxy {
    weight -30    # Reduce weight so sum stays positive
    ...
}

I generally recommend designing your weights so that all scripts failing together still leaves priority above zero on at least one node. Reserve FAULT state for situations where you truly want the node out of rotation.

Step 4: Verify Interface Tracking

If you're using track_interface, verify the interfaces are actually up:

ip link show

In your config:

vrrp_instance VI_1 {
    interface eth0
    track_interface {
        eth1 weight 10
        eth2
    }
}

An interface listed without a weight will cause FAULT if it goes down. An interface with a weight will reduce priority by that amount. Check that every interface you're tracking actually exists on this host with the exact name you've specified. Virtual machines and cloud instances especially love to rename interfaces (ens3, enp0s3, eth0 — never consistent).

# Verify interface names
ip -br link show

Step 5: Check for Authentication Mismatches

If you're using VRRP authentication (you probably shouldn't in modern setups, but many configs still use it), a mismatch between nodes won't cause FAULT directly — but it will cause the instance to never see valid advertisements, which can lead to unexpected behavior.

vrrp_instance VI_1 {
    authentication {
        auth_type PASS
        auth_pass yourpassword
    }
}

Make sure the auth_pass is identical on all nodes. Trailing whitespace in config files is a classic gotcha here.

Step 6: Enable Debug Logging

If the above steps haven't revealed the issue, turn up the verbosity:

# Add to /etc/sysconfig/keepalived or equivalent
KEEPALIVED_OPTIONS="-D -d -S 0"

Or run keepalived manually to watch output in real time:

keepalived -n -l -D

The -D flag dumps details about all state transitions, script execution results, and network events. This is particularly useful for watching the moment a transition to FAULT occurs.

A Practical Diagnostic Script

Here's a quick diagnostic script I keep on hand for Keepalived troubleshooting:

#!/bin/bash
# keepalived-diag.sh

echo "=== Keepalived Service Status ==="
systemctl status keepalived --no-pager

echo ""
echo "=== Current State Data ==="
kill -USR1 $(pidof keepalived) 2>/dev/null && sleep 1 && cat /tmp/keepalived.data 2>/dev/null || echo "keepalived not running"

echo ""
echo "=== Recent Log Entries ==="
journalctl -u keepalived -n 50 --no-pager

echo ""
echo "=== Interface Status ==="
ip -br link show

echo ""
echo "=== VIP Check ==="
ip addr show | grep -E "inet.*secondary|inet.*label"

echo ""
echo "=== Track Script Test ==="
CONFIG_FILE="${1:-/etc/keepalived/keepalived.conf}"
grep "script" "$CONFIG_FILE" | grep -v "#" | awk '{print $2}' | while read script; do
    echo -n "Testing: $script => "
    eval "$script" >/dev/null 2>&1
    echo "Exit code: $?"
done

Run it as:

chmod +x keepalived-diag.sh
sudo ./keepalived-diag.sh /etc/keepalived/keepalived.conf

Recovering From FAULT State

Once you've identified and fixed the underlying issue, you can either wait for Keepalived to detect the fix automatically (based on your rise count in track scripts) or force a reload:

systemctl reload keepalived

Note that reload sends a SIGHUP, which causes Keepalived to re-read configuration and restart VRRP instances. A restart is more disruptive and should be avoided on MASTER nodes if possible.

If you want to force a specific node to become MASTER (for testing), you can temporarily bump the priority via a config change and reload, or use the notify_master and notify_fault hooks to trigger automation.

Preventing Future FAULT State Surprises

A few practices that prevent you from ending up here again:

1. Test scripts in a restricted environment before deploying:

sudo -u keepalived env -i PATH=/usr/bin:/bin /your/script.sh
echo $?

2. Add explicit fall and rise counts to avoid flapping:

vrrp_script chk_app {
    script "/etc/keepalived/scripts/check_app.sh"
    interval 2
    fall 3      # Must fail 3 consecutive times
    rise 2      # Must succeed 2 consecutive times to recover
    weight -30
}

3. Use notify scripts to alert on state changes:

vrrp_instance VI_1 {
    notify_fault "/etc/keepalived/scripts/notify.sh FAULT"
    notify_master "/etc/keepalived/scripts/notify.sh MASTER"
    notify_backup "/etc/keepalived/scripts/notify.sh BACKUP"
}

4. Monitor the state actively — don't wait for users to tell you the VIP is gone. A simple check that alerts when an expected MASTER is in BACKUP or FAULT can save significant downtime.

Wrapping Up

FAULT state is actually Keepalived doing its job — it's removing a node it considers unhealthy from the VIP election. The frustration comes when the detection doesn't match your intent. Work through the diagnostic steps in order: check logs first, audit track scripts as the actual service user, verify your weight math can't go negative unexpectedly, and confirm interface names are correct.

Most FAULT state issues I've seen in production come down to track scripts working differently under keepalived's execution context than they do when run manually. Test in the same environment keepalived uses, and you'll catch the majority of these issues before they hit production.