Nginx Load Balancing: Round Robin, Least Conn, and IP Hash

Nginx's upstream module is one of the most production-tested load balancers available. It's fast, configurable, and handles the full spectrum from two backend nodes to hundreds. This guide covers all the built-in strategies, when to use each, and how to configure failover so your stack stays up when backends go down.

Upstream Block Basics

All load balancing configuration happens inside an upstream block in the http context:

# /etc/nginx/nginx.conf or /etc/nginx/conf.d/upstream.conf

upstream app_backend {
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}

server {
    listen 80;
    server_name app.example.com;

    location / {
        proxy_pass http://app_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Round Robin (Default)

Round robin is the default — no extra directive needed. Each request goes to the next server in the list, cycling through evenly.

upstream app_backend {
    # Round robin is implicit — no directive needed
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}

Weighted round robin — if your backends have different capacity:

upstream app_backend {
    server 10.0.1.10:8080 weight=3;  # Gets 3x as many requests
    server 10.0.1.11:8080 weight=2;
    server 10.0.1.12:8080 weight=1;
}

Request distribution with these weights: node1 gets 50%, node2 gets ~33%, node3 gets ~17%.

Best for: Stateless services with uniform request processing time and similar backend capacity.

Least Connections

Routes each new request to the backend currently handling the fewest active connections:

upstream app_backend {
    least_conn;

    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}

Supports weights too — a server with weight=2 can handle twice the connections before being passed over:

upstream app_backend {
    least_conn;

    server 10.0.1.10:8080 weight=2;  # Accepts up to 2x connections
    server 10.0.1.11:8080 weight=1;
}

Best for: Services with highly variable request duration (file uploads, database queries, video processing). Round robin can pile requests onto a slow backend; least_conn adapts automatically.

IP Hash

Routes the same client IP to the same backend every time:

upstream app_backend {
    ip_hash;

    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}

The hash is computed from the first three octets of the IPv4 address (to handle dynamic IPs in the same /24). For IPv6, the full address is used.

Best for: Applications with server-side session state that you haven't moved to a shared store like Redis yet. Not ideal for long-term — fix the architecture to be stateless instead.

Note: IP hash breaks even distribution if many users share a NAT IP. Monitor your backend load and switch to a session store + least_conn if you see imbalance.

Hash (Generic) — URL or Header-Based

The hash directive lets you hash on any Nginx variable:

# Sticky by URL — same URL always hits the same backend (good for caching)
upstream app_backend {
    hash $request_uri consistent;

    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}

# Sticky by session cookie
upstream app_backend {
    hash $cookie_sessionid consistent;

    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
}

# Sticky by custom header
upstream app_backend {
    hash $http_x_tenant_id consistent;

    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
}

The consistent parameter uses consistent hashing (ketama), which minimizes reshuffling when servers are added or removed — only ~1/N of keys remap instead of all of them.

Strategy Comparison

Strategy	Directive	Session Affinity	Even Distribution	Handles Slow Backends
Round robin	(default)	No	Yes (with equal weights)	No
Weighted RR	weight=N	No	Configurable	No
Least connections	least_conn	No	Adapts to load	Yes
IP hash	ip_hash	Yes (by IP)	Depends on IP diversity	No
Generic hash	hash $var	Yes (configurable)	Depends on hash distribution	No

Passive Health Checks and Failover

Nginx open-source uses passive health checks — it marks a backend as failed after it receives actual error responses. No separate health check connections are made.

upstream app_backend {
    server 10.0.1.10:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.12:8080 max_fails=3 fail_timeout=30s;
}

• max_fails=3: Mark the server down after 3 consecutive failures
• fail_timeout=30s: How long to mark it as unavailable, and also the window in which failures are counted

Combine with proxy error handling to define what counts as a failure:

location / {
    proxy_pass http://app_backend;

    # These response types count as failures for health tracking
    proxy_next_upstream error timeout http_500 http_502 http_503 http_504;

    # Try the next backend immediately on failure
    proxy_next_upstream_tries 2;
    proxy_next_upstream_timeout 10s;

    # Timeout settings
    proxy_connect_timeout 5s;
    proxy_read_timeout 30s;
    proxy_send_timeout 30s;
}

Backup Servers

Mark a server as a hot standby that only receives traffic when all primary servers are down:

upstream app_backend {
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080 backup;  # Only used if primary servers fail
}

Temporarily Removing a Server (Zero-Downtime Deploys)

Mark a server as down to take it out of rotation without removing it from config:

upstream app_backend {
    server 10.0.1.10:8080;
    server 10.0.1.11:8080 down;  # Excluded from rotation
    server 10.0.1.12:8080;
}

For live changes without restarting Nginx, use the upstream API (Nginx Plus) or reload config:

# Zero-downtime config reload
sudo nginx -t && sudo nginx -s reload

For dynamic upstream management without Nginx Plus, tools like lua-nginx-module or a separate control plane (Consul + consul-template) are common patterns.

Connection Persistence (keepalive to backends)

Opening a new TCP connection for every request adds latency. Configure keepalive to backend connections:

upstream app_backend {
    least_conn;

    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;

    # Keep up to 32 idle connections to backends in the pool
    keepalive 32;
}

location / {
    proxy_pass http://app_backend;

    # Required for keepalive to work properly
    proxy_http_version 1.1;
    proxy_set_header Connection "";
}

Monitoring Upstream Health

Enable the status module to watch upstream health in real time:

# Nginx stub status (available in open source)
location /nginx_status {
    stub_status on;
    allow 127.0.0.1;
    deny all;
}

# Watch key metrics
watch -n1 'curl -s http://localhost/nginx_status'

# Output format:
# Active connections: 45
# server accepts handled requests
#  1234 1234 5678
# Reading: 3 Writing: 12 Waiting: 30

For detailed upstream metrics, consider:

# Check error counts in logs
awk '$9 ~ /^5/' /var/log/nginx/access.log | \
  awk '{print $7}' | sort | uniq -c | sort -rn | head -20

# Latency analysis — if you log $upstream_response_time
awk '{print $NF}' /var/log/nginx/upstream.log | \
  sort -n | awk 'BEGIN{c=0} {a[c++]=$1} END{
    print "p50:", a[int(c*0.5)];
    print "p95:", a[int(c*0.95)];
    print "p99:", a[int(c*0.99)];
  }'

Production Upstream Logging

Add upstream timing to your log format for debugging:

log_format upstream_time '$remote_addr - $remote_user [$time_local] '
                         '"$request" $status $body_bytes_sent '
                         '"$http_referer" "$http_user_agent" '
                         'rt=$request_time uct=$upstream_connect_time '
                         'uht=$upstream_header_time urt=$upstream_response_time '
                         'ups=$upstream_addr';

server {
    access_log /var/log/nginx/access.log upstream_time;
}

Variables explained:

• $upstream_connect_time: Time to establish connection to backend
• $upstream_header_time: Time until first response byte received
• $upstream_response_time: Total time waiting for backend response
• $upstream_addr: Which backend actually handled the request

With these logs, you can identify slow backends, uneven distribution, and connection problems before they impact users.