networkingCritical
Fix: HAProxy Max Connection Limit Reached
New connections being refused or queued
!Symptoms
- New connections being refused or queued
- HAProxy stats showing maxconn reached
- Users experiencing connection timeouts
- Backend queue depth growing continuously
?Root Causes
- maxconn set too low for current traffic volume
- Backend servers responding slowly, holding connections open
- Connection leak — keep-alive connections not being released
- Sudden traffic spike (flash crowd, DDoS) exceeding capacity
- File descriptor limit (ulimit) lower than maxconn
#Diagnosis Steps
- 1Check HAProxy stats page: `curl http://localhost:9000/stats`
- 2Check current connections: `echo 'show info' | socat stdio /var/run/haproxy.sock`
- 3Check system file descriptors: `cat /proc/sys/fs/file-max` and `ulimit -n`
- 4Monitor connection rate: `echo 'show stat' | socat stdio /var/run/haproxy.sock`
- 5Check backend response times in HAProxy logs
>Fix
- 1Increase maxconn in haproxy.cfg global and frontend sections
- 2Increase system file descriptor limit: `ulimit -n 65536` and `/etc/security/limits.conf`
- 3Add more backend servers to distribute connections
- 4Enable HTTP keep-alive timeout to release idle connections: `timeout http-keep-alive 5s`
- 5Implement rate limiting to prevent connection exhaustion from abusive clients
*Prevention
- Set maxconn based on load testing, not guesses — test at 2x expected peak
- Monitor connection count and alert at 80% of maxconn
- Use connection queuing (server maxqueue) for graceful degradation
- Implement auto-scaling for backend servers
- Set appropriate timeouts (connect, client, server) to release stale connections
Related Error Messages
proxy reached process FD limitfrontend reached maxconnbackend has no server availableConnection refused