Nginx Upstream Load Balancing: Round-Robin, Least Connections, IP Hash, and Failover

Introduction

When your application outgrows a single backend server, Nginx's upstream load balancing becomes the backbone of your horizontal scaling strategy. Unlike DNS round-robin or hardware load balancers, Nginx gives you granular control over traffic distribution, failure detection, and connection pooling — all from a single configuration file on sw-infrarunbook-01.

This guide covers every major upstream load balancing method available in Nginx open source: round-robin, weighted distribution, least connections, IP hash, and generic hash. You will also learn how to configure passive health checks, backup servers, upstream keepalive connections, retry logic, and timeout tuning to build a production-grade load balancing layer for solvethenetwork.com.

Prerequisites

• Nginx installed on sw-infrarunbook-01 (Ubuntu 22.04 or AlmaLinux 9)
• Two or more backend application servers reachable within your RFC 1918 network
• DNS for solvethenetwork.com pointing to the Nginx server's public IP
• Root or sudo access on sw-infrarunbook-01

Understanding the Upstream Block

The

upstream

block lives inside the

http

context in your Nginx configuration. It defines a named group of backend servers that a

proxy_pass

directive can reference by name. The upstream block owns the routing logic — which server receives each request, how failures are tracked, and how connections are managed.

A minimal upstream block looks like this:

upstream app_backend {
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
    server 10.0.1.13:8080;
}

You reference it in a server block using

proxy_pass

:

server {
    listen 80;
    server_name solvethenetwork.com;

    location / {
        proxy_pass http://app_backend;
        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    }
}

With no additional directives, Nginx uses weighted round-robin by default — distributing requests evenly across all available servers in sequence.

Round-Robin Load Balancing (Default)

Round-robin is the implicit default. Nginx cycles through the server list sequentially: first request goes to

10.0.1.11

, second to

10.0.1.12

, third to

10.0.1.13

, then back to

10.0.1.11

. No extra directive is needed.

upstream app_backend {
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
    server 10.0.1.13:8080;
}

Round-robin works best when backends are homogeneous — same CPU, memory, and network throughput — and when individual requests have roughly equal processing cost. It is the simplest starting point and performs well for the majority of stateless API workloads.

Weighted Load Balancing

When backend servers have unequal capacity, the

weight

parameter directs proportionally more traffic to stronger nodes. A server with

weight=5

receives five times as many requests as one with

weight=1

.

upstream app_backend {
    server 10.0.1.11:8080 weight=5;
    server 10.0.1.12:8080 weight=3;
    server 10.0.1.13:8080 weight=1;
}

Out of every nine requests in this example,

10.0.1.11

handles five,

10.0.1.12

handles three, and

10.0.1.13

handles one. Weighted round-robin is particularly useful during rolling hardware upgrades where a newer, more powerful server should absorb the majority of traffic while a legacy node is being decommissioned.

Least Connections (

least_conn

)

Round-robin distributes requests evenly without regard to how long each takes. If some requests hold a backend connection open for several seconds — think file uploads, report exports, or slow database queries — that server accumulates a backlog while others sit idle. The

least_conn

directive routes each new request to the backend with the fewest active connections at that instant.

upstream app_backend {
    least_conn;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
    server 10.0.1.13:8080;
}

Use

least_conn

for any workload where request duration varies significantly. It pairs naturally with

weight

— a server with a higher weight receives new connections proportionally more often when connection counts are otherwise equal across the pool.

IP Hash (

ip_hash

)

When your application stores session state in server memory rather than in a shared external store, you need session persistence: the same client must always reach the same backend. The

ip_hash

directive hashes the client IP and pins that client to a specific upstream server for the duration of its session.

upstream app_backend {
    ip_hash;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
    server 10.0.1.13:8080;
}

For IPv4, Nginx hashes the first three octets, meaning all clients within the same /24 subnet resolve to the same backend. For IPv6, the full address is used. If the assigned backend becomes unavailable, Nginx automatically rehashes the client to a different live server.

Operational note:

ip_hash

cannot be combined with

least_conn

. For deployments that require both persistence and load-aware routing, the better architectural choice is to externalize session state to Redis and use

least_conn

without

ip_hash

.

Generic Hash (

hash

)

The

hash

directive lets you define an arbitrary Nginx variable as the hashing key. This enables consistent routing by request URI, cookie, query argument, or any combination. Routing the same URL to the same backend consistently maximizes in-process cache hit rates on each node.

upstream app_backend {
    hash $request_uri consistent;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
    server 10.0.1.13:8080;
}

The optional

consistent

keyword enables ketama-style consistent hashing, which minimizes key redistribution when a server is added to or removed from the pool. Without

consistent

, adding one server causes every hash bucket to recalculate, invalidating all existing routing assignments simultaneously.

Passive Health Checks and Failover

Nginx open source relies on passive health checking: it monitors real traffic responses rather than issuing synthetic probes. The

max_fails

and

fail_timeout

parameters control how quickly a troubled server is pulled from rotation and how long it remains out of service.

upstream app_backend {
    least_conn;
    server 10.0.1.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.12:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.13:8080 max_fails=3 fail_timeout=30s;
}

In this configuration, a backend is marked unavailable after three consecutive failures within a 30-second window. It stays out of rotation for 30 seconds before Nginx attempts to route a single probe request to it again. If that probe succeeds, the server re-enters the pool. The Nginx defaults are

max_fails=1

and

fail_timeout=10s

— raising

max_fails

to 3 prevents brief network blips from triggering unnecessary failovers.

Backup Servers

The

backup

parameter designates a server as a hot standby. Nginx routes no traffic to a backup server while at least one primary is alive. Only when every non-backup server is in a failed or

down

state does Nginx begin sending traffic to the backup.

upstream app_backend {
    least_conn;
    server 10.0.1.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.12:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.13:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.20:8080 backup;
}

Use a backup server for a degraded-mode endpoint that serves cached or simplified responses during a full primary outage, or for a geographically distant server that is too far away for normal traffic but acceptable as a last resort.

To temporarily drain a server without removing its configuration line — useful during maintenance windows — mark it

down

. Nginx will not route any traffic to it until

down

is removed and the config is reloaded:

upstream app_backend {
    server 10.0.1.11:8080;
    server 10.0.1.12:8080 down;
    server 10.0.1.13:8080;
}

Keepalive Connections to Upstream

By default Nginx opens a new TCP connection to the upstream for every proxied request and tears it down afterward. At high request rates this creates significant overhead from repeated TCP three-way handshakes and TIME_WAIT socket accumulation. The

keepalive

directive instructs each Nginx worker process to maintain a cache of idle persistent connections to the upstream group.

upstream app_backend {
    least_conn;
    server 10.0.1.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.12:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.13:8080 max_fails=3 fail_timeout=30s;
    keepalive 32;
}

The value

32

sets the maximum number of idle keepalive connections cached per worker process for this upstream group. When all cached connections are in use, Nginx opens new connections normally. For keepalive to function correctly, the proxy location block must also be configured as follows:

location / {
    proxy_pass         http://app_backend;
    proxy_http_version 1.1;
    proxy_set_header   Connection "";
    proxy_set_header   Host             $host;
    proxy_set_header   X-Real-IP        $remote_addr;
    proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
}

Setting

proxy_http_version 1.1

is required because HTTP/1.0 does not support persistent connections. Clearing the

Connection

header with

proxy_set_header Connection ""

prevents Nginx from forwarding a

Connection: close

header from the client to the upstream, which would force the upstream to terminate the connection after every response.

Proxy Timeout Configuration

Three directives govern how long Nginx waits at each phase of an upstream interaction:

• proxy_connect_timeout — Maximum time to establish the TCP connection to the upstream. The default is 60 seconds; for backends on the same LAN, 5 seconds is more appropriate and enables faster failover.
• proxy_send_timeout — Maximum idle time between two successive write operations to the upstream. Resets with each write. Default is 60 seconds.
• proxy_read_timeout — Maximum idle time between two successive read operations from the upstream. This is not the total request duration but the inter-packet gap. Default is 60 seconds.

location / {
    proxy_pass http://app_backend;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_set_header Host             $host;
    proxy_set_header X-Real-IP        $remote_addr;
    proxy_set_header X-Forwarded-For  $proxy_add_x_forwarded_for;

    proxy_connect_timeout  5s;
    proxy_send_timeout    60s;
    proxy_read_timeout    60s;
}

For long-running backend operations such as PDF generation or large data exports, raise

proxy_read_timeout

appropriately. For fast internal microservice calls, keep

proxy_connect_timeout

short so Nginx detects a downed backend within seconds and retries on another node.

Retry Logic with proxy_next_upstream

The

proxy_next_upstream

directive defines the conditions under which Nginx retries a failed request on the next available server. This complements passive health checks by handling in-flight request failures rather than server-level failures.

location / {
    proxy_pass http://app_backend;
    proxy_http_version 1.1;
    proxy_set_header Connection "";

    proxy_next_upstream      error timeout invalid_header http_500 http_502 http_503;
    proxy_next_upstream_tries 2;
    proxy_next_upstream_timeout 10s;
}

proxy_next_upstream_tries 2

limits the retry chain to two attempts total, preventing a slow backend from forcing the request to be tried against every server in the pool before returning an error to the client. Be conservative when enabling retry for

http_500

on non-idempotent HTTP methods such as POST — retrying a payment submission or database write could cause duplicate side effects. To restrict retries to safe methods only, omit

http_500

from the directive values and handle application-level 500 errors separately.

Complete Production Configuration

The following is a full production-ready configuration for sw-infrarunbook-01, load balancing HTTPS traffic to three application backends serving solvethenetwork.com using least connections, keepalive, passive health checks, a backup server, and retry logic.

# /etc/nginx/conf.d/solvethenetwork.conf

upstream app_backend {
    least_conn;

    server 10.0.1.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.12:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.13:8080 max_fails=3 fail_timeout=30s;

    # Hot standby: receives traffic only when all primaries are down
    server 10.0.1.20:8080 backup;

    keepalive 32;
}

server {
    listen 80;
    server_name solvethenetwork.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name solvethenetwork.com;

    ssl_certificate      /etc/letsencrypt/live/solvethenetwork.com/fullchain.pem;
    ssl_certificate_key  /etc/letsencrypt/live/solvethenetwork.com/privkey.pem;
    ssl_protocols        TLSv1.2 TLSv1.3;
    ssl_ciphers          HIGH:!aNULL:!MD5;

    access_log  /var/log/nginx/solvethenetwork_access.log;
    error_log   /var/log/nginx/solvethenetwork_error.log warn;

    location / {
        proxy_pass         http://app_backend;
        proxy_http_version 1.1;
        proxy_set_header   Connection        "";
        proxy_set_header   Host              $host;
        proxy_set_header   X-Real-IP         $remote_addr;
        proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;

        proxy_connect_timeout   5s;
        proxy_send_timeout     60s;
        proxy_read_timeout     60s;

        proxy_next_upstream      error timeout http_502 http_503;
        proxy_next_upstream_tries 2;
    }

    # Lightweight liveness check served by Nginx itself
    location /nginx-health {
        access_log off;
        return 200 "OK\n";
        add_header Content-Type text/plain;
    }
}

Testing and Validation

Test the configuration syntax on sw-infrarunbook-01 before applying it:

sudo nginx -t

Expected output on success:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Reload Nginx gracefully, without dropping active connections:

sudo systemctl reload nginx

Verify distribution across all backends by sending a batch of requests and checking each backend's access log:

for i in {1..12}; do curl -s -o /dev/null -w "%{http_code}\n" https://solvethenetwork.com/; done

Each of the three primary backends should show roughly four entries in its access log for the twelve requests sent. With

least_conn

, distribution may not be perfectly even if some backends are already serving open connections when the test runs — this is expected and correct behavior.

Monitoring with stub_status

Nginx open source includes a basic status module that exposes aggregate connection and request counters. Enable it on a loopback-only listener so it is never reachable from the internet:

server {
    listen 127.0.0.1:8081;

    location /nginx_status {
        stub_status;
        allow 127.0.0.1;
        deny  all;
    }
}

curl http://127.0.0.1:8081/nginx_status

Sample output:

Active connections: 47
server accepts handled requests
 142680 142680 389201
Reading: 0 Writing: 5 Waiting: 42

For per-upstream metrics — request rate per backend, active connections per server, upstream error rates — consider the open-source

nginx-module-vts

module or the

nginx-prometheus-exporter

sidecar to feed data into a Prometheus and Grafana stack. Nginx Plus provides these natively through its upstream health check API, but the open-source path covered here is sufficient for the majority of production deployments.

---

Related Articles

• [HAProxy] HAProxy Load Balancing Algorithms: Round-Robin, Leastconn, Source Hashing, URI Hashing, and Random
• [HAProxy] HAProxy vs Nginx: Which Is Better for Load Balancing
• [Nginx] What Is Reverse Proxy vs Forward Proxy Explained
• [Nginx] Nginx SSL Handshake Failure