Keepalived and VRRP: Building High-Availability Failover for Linux Services

What Keepalived Solves

A single load balancer is a single point of failure. Keepalived uses VRRP (Virtual Router Redundancy Protocol) to share a virtual IP between two or more servers. The master node owns the VIP; if it fails, a backup takes over within seconds — transparently from the perspective of clients hitting the VIP.

This is the standard pattern for making Nginx, HAProxy, or any TCP service highly available without a hardware load balancer.

---

Installation

# RHEL/Rocky/AlmaLinux
sudo dnf install keepalived -y

# Ubuntu/Debian
sudo apt install keepalived -y

sudo systemctl enable keepalived

---

Basic VRRP Configuration

Master Node

Edit /etc/keepalived/keepalived.conf on the master:

global_defs {
    router_id LB01          # unique name for this node
    script_user root
    enable_script_security
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0          # NIC that carries the VIP
    virtual_router_id 51    # must match on all nodes in the group (1-255)
    priority 100            # higher = preferred; master should be highest
    advert_int 1            # VRRP advertisement interval (seconds)

    authentication {
        auth_type PASS
        auth_pass secretpass    # shared secret between nodes
    }

    virtual_ipaddress {
        192.168.1.100/24 dev eth0   # the virtual IP
    }
}

Backup Node

Edit /etc/keepalived/keepalived.conf on the backup:

global_defs {
    router_id LB02
    script_user root
    enable_script_security
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51    # must match master
    priority 90             # lower than master
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass secretpass
    }

    virtual_ipaddress {
        192.168.1.100/24 dev eth0
    }
}

Start on both nodes:

sudo systemctl start keepalived

Verify the VIP is assigned to the master:

ip addr show eth0
# You should see 192.168.1.100/24 on the master node

---

Failover Behavior

1. Master broadcasts VRRP advertisements every advert_int seconds
2. Backup listens; if it doesn't hear from master for 3×advert_int seconds, it takes over
3. Backup claims the VIP (brings it up on its own interface)
4. When master recovers, it reclaims the VIP (MASTER state + higher priority)

To prevent "flapping" (rapid failover/failback), set nopreempt on the backup config to prevent the master from taking back the VIP after recovery:

vrrp_instance VI_1 {
    state BACKUP
    nopreempt           # don't preempt the current master on recovery
    ...
}

And change the master's state to BACKUP with higher priority — both nodes are BACKUP, but the one with higher priority wins initially.

---

Health Check Scripts

VRRP failover alone is based on network connectivity. For application-level failover, use vrrp_script to check if the service is actually working:

vrrp_script check_haproxy {
    script "/usr/bin/pgrep haproxy"     # returns 0 if haproxy is running
    interval 2                           # check every 2 seconds
    weight -20                           # subtract 20 from priority if fails
    fall 2                               # fail after 2 consecutive failures
    rise 2                               # recover after 2 consecutive successes
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass secretpass
    }

    virtual_ipaddress {
        192.168.1.100/24
    }

    track_script {
        check_haproxy
    }
}

If HAProxy is down: priority drops from 100 to 80 (100 - 20). The backup at priority 90 wins and takes the VIP.

Custom Health Check Script

# /usr/local/bin/check_nginx.sh
#!/bin/bash
if curl -sf http://127.0.0.1/health > /dev/null 2>&1; then
    exit 0    # healthy — script succeeds
else
    exit 1    # unhealthy — triggers weight adjustment
fi

vrrp_script check_nginx {
    script "/usr/local/bin/check_nginx.sh"
    interval 3
    weight -30
    fall 2
    rise 2
}

Make scripts executable:

sudo chmod +x /usr/local/bin/check_nginx.sh

---

Multiple VIPs / Active-Active

Run multiple VRRP instances for different VIPs — each can have a different master, achieving active-active load distribution:

# On LB01: master for VIP1, backup for VIP2
vrrp_instance VI_1 {
    state MASTER
    virtual_router_id 51
    priority 100
    virtual_ipaddress {
        192.168.1.100/24
    }
}

vrrp_instance VI_2 {
    state BACKUP
    virtual_router_id 52
    priority 90
    virtual_ipaddress {
        192.168.1.101/24
    }
}

# On LB02: backup for VIP1, master for VIP2
vrrp_instance VI_1 {
    state BACKUP
    virtual_router_id 51
    priority 90
    virtual_ipaddress {
        192.168.1.100/24
    }
}

vrrp_instance VI_2 {
    state MASTER
    virtual_router_id 52
    priority 100
    virtual_ipaddress {
        192.168.1.101/24
    }
}

DNS round-robin between both VIPs completes the active-active setup.

---

Notify Scripts

Run a script when state changes (e.g., send an alert):

vrrp_instance VI_1 {
    ...
    notify_master "/usr/local/bin/notify.sh MASTER"
    notify_backup "/usr/local/bin/notify.sh BACKUP"
    notify_fault  "/usr/local/bin/notify.sh FAULT"
}

# /usr/local/bin/notify.sh
#!/bin/bash
STATE=$1
echo "$(date): Keepalived state changed to $STATE on $(hostname)" >> /var/log/keepalived-state.log
# Add Slack/PagerDuty notification here

---

Monitoring and Troubleshooting

# Check Keepalived status
sudo systemctl status keepalived

# View logs
sudo journalctl -u keepalived -f

# Check current VIP assignment
ip addr show | grep "192.168.1.100"

# Monitor VRRP advertisements with tcpdump
sudo tcpdump -i eth0 vrrp

# Test failover: stop keepalived on master
sudo systemctl stop keepalived
# VIP should move to backup within ~3 seconds

# Verify from a client
ping 192.168.1.100   # should continue with minor blip during failover

---

Common Pitfalls

VIP not moving on failover: Check that virtual_router_id matches on both nodes and auth_pass is identical.

Both nodes claim MASTER: Network split-brain — VRRP advertisements aren't reaching the other node. Check firewall rules for VRRP (protocol 112, multicast 224.0.0.18):

# RHEL/Rocky
sudo firewall-cmd --permanent --add-protocol=vrrp
sudo firewall-cmd --reload

# Ubuntu (ufw)
sudo ufw allow proto vrrp

Health check not triggering failover: Ensure the script file is executable and returns a non-zero exit code on failure. Test manually: sudo /usr/local/bin/check_nginx.sh; echo $?

Keepalived requires privileges: With enable_script_security, scripts must be owned by root and not world-writable.