HAProxy + Keepalived: Production HA Load Balancer Setup

The Standard HA Load Balancer Pattern

HAProxy alone is single-point-of-failure. Keepalived alone doesn't load-balance. Together, they form the most common open-source HA load balancer architecture:

• Keepalived manages a virtual IP (VIP) that floats between two servers
• HAProxy on each server handles the actual load balancing
• Clients connect to the VIP — they never know which HAProxy is active

This guide builds a complete setup: two load balancer nodes, one VIP, HAProxy distributing to three backend web servers.

---

Environment

VIP:   192.168.1.100  (shared, managed by Keepalived)
LB01:  192.168.1.10   (master)
LB02:  192.168.1.11   (backup)

Backends:
  web01: 192.168.1.20:80
  web02: 192.168.1.21:80
  web03: 192.168.1.22:80

---

Step 1: Install HAProxy and Keepalived on Both Nodes

# RHEL/Rocky
sudo dnf install haproxy keepalived -y

# Ubuntu
sudo apt install haproxy keepalived -y

sudo systemctl enable haproxy keepalived

---

Step 2: Configure HAProxy (Identical on Both Nodes)

Edit /etc/haproxy/haproxy.cfg:

global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    maxconn 50000

defaults
    log global
    mode http
    option httplog
    option dontlognull
    option forwardfor
    option http-server-close
    timeout connect 5s
    timeout client  50s
    timeout server  50s
    retries 3

# Stats page
frontend stats
    bind *:8404
    stats enable
    stats uri /stats
    stats refresh 10s
    stats auth admin:strongpassword
    stats hide-version

# HTTP frontend
frontend http_front
    bind *:80
    # Redirect to HTTPS
    redirect scheme https code 301 if !{ ssl_fc }

# HTTPS frontend
frontend https_front
    bind *:443 ssl crt /etc/haproxy/certs/example.com.pem alpn h2,http/1.1
    default_backend web_servers

    # X-Forwarded headers
    http-request set-header X-Forwarded-Proto https
    http-request set-header X-Real-IP %[src]

    # Rate limiting by source IP
    stick-table type ip size 100k expire 30s store conn_cur,conn_rate(30s)
    http-request track-sc0 src
    http-request deny deny_status 429 if { sc_conn_rate(0) gt 100 }

# Backend pool
backend web_servers
    balance roundrobin
    option httpchk GET /health HTTP/1.1\r\nHost:\ example.com

    # Servers with health checks
    server web01 192.168.1.20:80 check inter 5s fall 3 rise 2
    server web02 192.168.1.21:80 check inter 5s fall 3 rise 2
    server web03 192.168.1.22:80 check inter 5s fall 3 rise 2 backup

Combine certificate + key for HAProxy:

cat /etc/letsencrypt/live/example.com/fullchain.pem \
    /etc/letsencrypt/live/example.com/privkey.pem \
    > /etc/haproxy/certs/example.com.pem
chmod 600 /etc/haproxy/certs/example.com.pem

Test and start:

sudo haproxy -c -f /etc/haproxy/haproxy.cfg
sudo systemctl start haproxy

---

Step 3: Configure Keepalived

LB01 (Master) — /etc/keepalived/keepalived.conf

global_defs {
    router_id LB01
    script_user root
    enable_script_security
}

vrrp_script check_haproxy {
    script "pgrep haproxy"
    interval 2
    weight -30
    fall 2
    rise 2
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass vrrppass
    }

    virtual_ipaddress {
        192.168.1.100/24 dev eth0 label eth0:vip
    }

    track_script {
        check_haproxy
    }

    notify_master "/usr/local/bin/haproxy-notify.sh MASTER"
    notify_backup "/usr/local/bin/haproxy-notify.sh BACKUP"
    notify_fault  "/usr/local/bin/haproxy-notify.sh FAULT"
}

LB02 (Backup) — /etc/keepalived/keepalived.conf

global_defs {
    router_id LB02
    script_user root
    enable_script_security
}

vrrp_script check_haproxy {
    script "pgrep haproxy"
    interval 2
    weight -30
    fall 2
    rise 2
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 90
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass vrrppass
    }

    virtual_ipaddress {
        192.168.1.100/24 dev eth0 label eth0:vip
    }

    track_script {
        check_haproxy
    }

    notify_master "/usr/local/bin/haproxy-notify.sh MASTER"
    notify_backup "/usr/local/bin/haproxy-notify.sh BACKUP"
    notify_fault  "/usr/local/bin/haproxy-notify.sh FAULT"
}

Notify Script

cat > /usr/local/bin/haproxy-notify.sh << 'EOF'
#!/bin/bash
STATE=$1
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
echo "$TIMESTAMP: Keepalived state transition to $STATE on $(hostname)" >> /var/log/keepalived-transitions.log
# Optionally send alert: curl Slack webhook, etc.
EOF
chmod +x /usr/local/bin/haproxy-notify.sh

Allow VRRP through the firewall:

# RHEL/Rocky
sudo firewall-cmd --permanent --add-protocol=vrrp && sudo firewall-cmd --reload

# Ubuntu
sudo ufw allow proto vrrp

Start Keepalived:

sudo systemctl start keepalived

---

Step 4: Verify the Setup

# On LB01: VIP should be present
ip addr show eth0 | grep "192.168.1.100"

# Test from a client
curl -I https://192.168.1.100/

# Check HAProxy stats
curl -u admin:strongpassword http://192.168.1.100:8404/stats

# Watch VRRP traffic
sudo tcpdump -i eth0 vrrp -n

---

Step 5: Test Failover

# Simulate LB01 failure
sudo systemctl stop haproxy   # on LB01

# Watch on LB02 — it should claim the VIP
ip addr show eth0   # LB02 should now show 192.168.1.100
sudo journalctl -u keepalived -f

# Test connectivity is maintained
curl -I https://192.168.1.100/   # should still work from clients

# Restore LB01
sudo systemctl start haproxy   # LB01 reclaims VIP due to higher priority

Expected failover time: 2-4 seconds (2× advert_int + script check interval).

---

Logging and Monitoring

# HAProxy log (via rsyslog)
sudo tail -f /var/log/haproxy.log

# HAProxy socket stats
echo "show info" | sudo socat stdio /run/haproxy/admin.sock
echo "show stat" | sudo socat stdio /run/haproxy/admin.sock | cut -d',' -f1,2,18,19

# Keepalived logs
sudo journalctl -u keepalived -n 50

# Combined health view
watch -n2 "ip addr show eth0 | grep 192.168.1.100; echo '---'; echo 'show stat' | socat stdio /run/haproxy/admin.sock | cut -d, -f1,2,18,19,20"

---

HAProxy Runtime API: Zero-Downtime Backend Management

# Disable a backend server (drain connections gracefully)
echo "set server web_servers/web03 state drain" | sudo socat stdio /run/haproxy/admin.sock

# Take a server offline
echo "set server web_servers/web03 state maint" | sudo socat stdio /run/haproxy/admin.sock

# Bring it back
echo "set server web_servers/web03 state ready" | sudo socat stdio /run/haproxy/admin.sock

# Check current server states
echo "show servers state" | sudo socat stdio /run/haproxy/admin.sock

This combination — HAProxy handling distribution and Keepalived handling VIP failover — provides a robust, production-grade load balancer without any commercial components.