Keepalived VRRP: Build a Highly Available Load Balancer Pair

Running a single load balancer is a single point of failure. When it crashes — and it will crash — your entire application goes down with it. Keepalived solves this by running two load balancer nodes sharing a virtual IP address (VIP). When the primary fails, the secondary takes the VIP in seconds. Users see a brief blip; the application stays up.

Architecture Overview

The setup:

Internet → Virtual IP: 10.0.0.100 (the address DNS points to)
                    |
        +-----------+-----------+
        |                       |
  LB-Primary: 10.0.0.10   LB-Secondary: 10.0.0.20
  (MASTER — owns VIP)      (BACKUP — monitoring)
        |                       |
        +-----------+-----------+
                    |
           Backend servers: 10.0.1.x

Both nodes run Nginx or HAProxy. Keepalived runs VRRP between them. The MASTER holds the VIP; the BACKUP listens for VRRP advertisements and takes the VIP if the MASTER goes silent.

Install Keepalived

# Debian/Ubuntu
sudo apt install keepalived

# RHEL/CentOS/Rocky
sudo dnf install keepalived

# Verify version
keepalived --version

Non-Local IP Binding

Your load balancer (Nginx/HAProxy) needs to bind to the VIP, which might not exist on this node when it starts. Enable non-local binding:

# Make permanent
echo "net.ipv4.ip_nonlocal_bind = 1" >> /etc/sysctl.conf
sysctl -p

# Verify
sysctl net.ipv4.ip_nonlocal_bind

Primary Node Configuration (MASTER)

# /etc/keepalived/keepalived.conf on LB-Primary (10.0.0.10)

global_defs {
    # Unique identifier for this node
    router_id LB_PRIMARY

    # Send email alerts on state transitions
    notification_email {
        [email protected]
    }
    notification_email_from [email protected]
    smtp_server 127.0.0.1
    smtp_connect_timeout 30

    # Enable script security (required for track_script)
    script_user root
    enable_script_security
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0              # Network interface to use for VRRP
    virtual_router_id 51        # Must match on both nodes (1-255)
    priority 100                # Higher = preferred master
    advert_int 1                # VRRP advertisement interval (seconds)

    # Authenticate VRRP packets to prevent rogue nodes
    authentication {
        auth_type PASS
        auth_pass strongpassword
    }

    # The VIP — this IP will float between nodes
    virtual_ipaddress {
        10.0.0.100/24 dev eth0 label eth0:vip
    }

    # Run scripts on state transitions
    notify_master   "/etc/keepalived/notify.sh MASTER"
    notify_backup   "/etc/keepalived/notify.sh BACKUP"
    notify_fault    "/etc/keepalived/notify.sh FAULT"
}

Secondary Node Configuration (BACKUP)

# /etc/keepalived/keepalived.conf on LB-Secondary (10.0.0.20)

global_defs {
    router_id LB_SECONDARY
    notification_email {
        [email protected]
    }
    notification_email_from [email protected]
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    script_user root
    enable_script_security
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51        # Must match primary
    priority 90                 # Lower than primary
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass strongpassword  # Must match primary
    }

    virtual_ipaddress {
        10.0.0.100/24 dev eth0 label eth0:vip
    }

    notify_master   "/etc/keepalived/notify.sh MASTER"
    notify_backup   "/etc/keepalived/notify.sh BACKUP"
    notify_fault    "/etc/keepalived/notify.sh FAULT"
}

Notification Script

#!/bin/bash
# /etc/keepalived/notify.sh
STATE=$1
HOSTNAME=$(hostname)
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')

case $STATE in
    MASTER)
        echo "$TIMESTAMP: $HOSTNAME became MASTER (now owns VIP)" \
          >> /var/log/keepalived-transitions.log
        # Optional: send alert via Slack/PagerDuty webhook
        # curl -s -X POST https://hooks.slack.com/services/... \
        #   -H 'Content-type: application/json' \
        #   --data "{\"text\":\"$HOSTNAME became KEEPALIVED MASTER\"}"
        ;;
    BACKUP)
        echo "$TIMESTAMP: $HOSTNAME became BACKUP (released VIP)" \
          >> /var/log/keepalived-transitions.log
        ;;
    FAULT)
        echo "$TIMESTAMP: $HOSTNAME entered FAULT state" \
          >> /var/log/keepalived-transitions.log
        ;;
esac

chmod +x /etc/keepalived/notify.sh

Start and Verify

# Enable and start on both nodes
sudo systemctl enable keepalived
sudo systemctl start keepalived

# Check status
sudo systemctl status keepalived

# Verify VIP is on the primary
ip addr show eth0
# Look for: inet 10.0.0.100/24 scope global secondary eth0:vip

# Verify from the secondary — VIP should NOT be here
# ip addr show eth0 | grep 10.0.0.100  (should return nothing)

Test from a remote machine:

ping -c 3 10.0.0.100
# Should respond from primary node

# Verify with ARP — see which MAC owns the VIP
arping -I eth0 10.0.0.100 -c 3

Testing Failover

This is the critical test — never assume it works until you've verified it:

# Start a continuous ping from a client to the VIP
ping 10.0.0.100

# In another terminal on the PRIMARY node:
sudo systemctl stop keepalived
# Or simulate a crash:
# sudo killall -9 keepalived

# Watch the ping output — you should see 1-3 dropped packets,
# then responses resume from the SECONDARY

Expected ping output during failover:

64 bytes from 10.0.0.100: icmp_seq=42 ttl=64 time=0.4 ms
64 bytes from 10.0.0.100: icmp_seq=43 ttl=64 time=0.3 ms
Request timeout for icmp_seq 44           <- Failover window
64 bytes from 10.0.0.100: icmp_seq=45 ttl=64 time=0.5 ms
64 bytes from 10.0.0.100: icmp_seq=46 ttl=64 time=0.4 ms

With advert_int 1 and fall 3 (VRRP default), failover completes in approximately 3 seconds.

Preemption Control

By default, when the MASTER comes back online after a failure, it reclaims the VIP immediately (preemptive behavior). This causes a second failover — often unnecessary:

# Disable preemption on the primary to prevent unnecessary failback
vrrp_instance VI_1 {
    state MASTER
    nopreempt                   # Add this line
    priority 100
    ...
}

With nopreempt, the secondary keeps the VIP after a failover until it fails or you manually trigger a transition. This is usually the right behavior for production.

Active-Active with Two VIPs

For active-active load balancing across both nodes, create two VRRP instances where each node is MASTER for one VIP:

# LB-Primary: MASTER for VIP1, BACKUP for VIP2
vrrp_instance VI_1 {
    state MASTER
    priority 100
    virtual_router_id 51
    virtual_ipaddress { 10.0.0.100/24 }
}

vrrp_instance VI_2 {
    state BACKUP
    priority 90
    virtual_router_id 52
    virtual_ipaddress { 10.0.0.101/24 }
}

# LB-Secondary: BACKUP for VIP1, MASTER for VIP2
vrrp_instance VI_1 {
    state BACKUP
    priority 90
    virtual_router_id 51
    virtual_ipaddress { 10.0.0.100/24 }
}

vrrp_instance VI_2 {
    state MASTER
    priority 100
    virtual_router_id 52
    virtual_ipaddress { 10.0.0.101/24 }
}

Point half your DNS records to each VIP. Both nodes share the load in normal operation; either can handle everything if the other fails.

Monitoring VRRP State

# Real-time keepalived log watching
journalctl -u keepalived -f

# Check current state
cat /var/run/keepalived/keepalived.pid  # exists if running

# VRRP state from logs
grep "VRRP_Instance" /var/log/syslog | tail -20

# Network-level VRRP packet capture (protocol 112)
sudo tcpdump -i eth0 vrrp -n
# Output: IP 10.0.0.10 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 100

If you see both nodes claiming MASTER state simultaneously (split-brain), check:

1. Network connectivity between the nodes on the VRRP interface
2. That virtual_router_id values match
3. That auth_pass values match exactly
4. Firewall rules — VRRP uses protocol 112, not TCP or UDP