Nginx Caching Proxy Setup and Tuning

Prerequisites

Before you start, make sure you have Nginx 1.18 or newer installed. The proxy cache functionality is baked into

ngx_http_proxy_module

, which ships compiled into the binary by default on every major distribution. You can quickly confirm it's present by running

nginx -V 2>&1 | grep proxy

— if you see

--with-http_proxy_module

or nothing at all about it being excluded, you're good. It's included unless someone explicitly stripped it out.

You'll also need the following in place before touching a config file:

• A backend upstream application server — in this guide that's 192.168.10.20:8080
• The proxy server itself — we'll call it sw-infrarunbook-01, running Ubuntu 22.04 LTS
• Dedicated disk space for the cache storage path. Don't share this with your root partition. I've seen systems fall over hard when a runaway cache filled

  /var

  on a root disk and the OS couldn't write logs or temp files. Mount a separate volume.
• Root or sudo access to edit configs and create directories under

  /data

The configuration examples below work identically on RHEL 9 and Rocky Linux — just swap

apt

for

dnf

and

www-data

for

nginx

in the ownership commands.

---

How Nginx Proxy Caching Actually Works

It's worth spending two minutes on the mechanics before writing a single line of config, because misunderstanding this is where most of the common mistakes originate.

Nginx's proxy cache stores full upstream HTTP responses on disk and serves them to subsequent clients without touching the backend. Each cached object is keyed by a string you define — by default it's built from the scheme, host, and request URI. When a request arrives, Nginx computes the cache key, looks up the key in a shared memory zone, and either serves the on-disk cached content (HIT) or proxies the request upstream and stores the response (MISS).

The shared memory zone does not store the response bodies. It stores metadata: key hashes, expiry timestamps, and pointers to on-disk files. The actual content lives on disk at the path you specify. This matters for sizing — you're sizing the metadata index, not the cache content itself. One megabyte of shared memory holds metadata for roughly 8,000 cached objects.

Keep this separation in mind throughout. When someone says their cache "is full," they might mean the disk path hit

max_size

, or they might mean the keys zone exhausted its shared memory. Those are different problems with different fixes.

---

Step-by-Step Setup

Step 1: Install Nginx

On Ubuntu 22.04, the mainline repository gives you the most current stable release:

apt update && apt install -y nginx
nginx -v
# nginx version: nginx/1.24.0

If you need a newer version than what's in the default repos, add the official Nginx package repository before installing. The version matters less for caching than for some other features, but anything below 1.14 is missing useful directives like

proxy_cache_lock_timeout

and

use_temp_path

.

Step 2: Create the Cache Directory

Pick a dedicated path with its own storage. We'll use

/data/nginx/cache

. Create it and set correct ownership so Nginx's worker processes can write to it:

mkdir -p /data/nginx/cache
chown -R www-data:www-data /data/nginx/cache
chmod 700 /data/nginx/cache

On a RHEL-based system with SELinux enforcing, you'll need to set the correct file context or the workers will get permission-denied errors even with correct POSIX ownership:

semanage fcontext -a -t httpd_cache_t "/data/nginx/cache(/.*)?" 
restorecon -Rv /data/nginx/cache

Step 3: Define the Cache Zone

The

proxy_cache_path

directive belongs in the

http

block — not inside a

server

or

location

block. This is where a surprising number of people stumble. Nginx will refuse to start if you place it at the wrong scope level.

Open

/etc/nginx/nginx.conf

and add this inside the

http { }

block:

proxy_cache_path /data/nginx/cache
    levels=1:2
    keys_zone=STATIC_CACHE:50m
    max_size=20g
    inactive=60m
    use_temp_path=off;

Every one of these parameters is load-bearing. Here's what each one actually does:

• levels=1:2 — creates a two-level directory hierarchy under the cache path. Without this, every cached file lands in a single flat directory. On Linux, directory lookups in flat directories with tens of thousands of files become progressively slower due to how dentries are traversed. The two-level split keeps each subdirectory small.
• keys_zone=STATIC_CACHE:50m — names the zone

  STATIC_CACHE

  (referenced later in location blocks) and allocates 50MB of shared memory for metadata. At ~8,000 entries per megabyte, 50MB covers roughly 400,000 cached objects — adequate for most deployments.
• max_size=20g — hard cap on total disk usage for this cache path. When Nginx hits this limit, it evicts the least recently used entries via its cache manager process.
• inactive=60m — any cached object not accessed within 60 minutes becomes eligible for eviction regardless of its TTL. This prevents stale content from accumulating for infrequently requested URLs.
• use_temp_path=off — instructs Nginx to write incoming responses directly to the final cache path instead of writing to a temp directory first. Without this, you pay for an unnecessary file rename (or worse, a full copy if temp and cache are on different filesystems) for every single cache MISS.

Step 4: Configure the Proxy Virtual Host

Now the server block. Start with the basic structure before adding the tuning layers:

upstream backend_app {
    server 192.168.10.20:8080;
    keepalive 32;
}

server {
    listen 80;
    server_name solvethenetwork.com www.solvethenetwork.com;

    access_log /var/log/nginx/solvethenetwork_access.log;
    error_log  /var/log/nginx/solvethenetwork_error.log warn;

    location / {
        proxy_pass         http://backend_app;
        proxy_cache        STATIC_CACHE;
        proxy_cache_valid  200 302 10m;
        proxy_cache_valid  404     1m;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        proxy_cache_lock   on;

        proxy_set_header   Host              $host;
        proxy_set_header   X-Real-IP         $remote_addr;
        proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;

        add_header         X-Cache-Status    $upstream_cache_status;
    }
}

The

X-Cache-Status

response header is something I add to every caching proxy I build. It exposes

$upstream_cache_status

directly to the client, so you can instantly see from a

curl -I

whether you're getting a HIT, MISS, BYPASS, EXPIRED, or STALE response. Invaluable for debugging, and cheap to add.

The

proxy_cache_use_stale

directive is doing important work here too. When the backend returns a 500 or is timing out, Nginx will serve the last cached version of the content instead of passing the error to the client. That's a meaningful resilience improvement at essentially zero cost.

Step 5: Add Cache Bypass Logic

You almost certainly don't want to cache everything. Authenticated sessions, shopping carts, API mutation endpoints — none of these should be cached. The right way to handle this in Nginx is with

map

blocks that define bypass conditions, rather than sprinkling

if

directives through your location blocks. Put these in the

http

block alongside the cache path definition:

map $http_cookie $no_cache_cookie {
    default         0;
    "~*session_id"  1;
    "~*auth_token"  1;
}

map $request_method $no_cache_method {
    default  0;
    POST     1;
    PUT      1;
    DELETE   1;
    PATCH    1;
}

Then reference both variables inside the location block:

        proxy_cache_bypass  $no_cache_cookie $no_cache_method;
        proxy_no_cache      $no_cache_cookie $no_cache_method;

The distinction between

proxy_cache_bypass

and

proxy_no_cache

trips people up, so it's worth being explicit.

proxy_cache_bypass

controls whether Nginx reads from cache — when it's set, Nginx goes upstream even if a cached copy exists.

proxy_no_cache

controls whether Nginx writes to cache — when it's set, Nginx won't store the upstream response. Setting both together means: don't serve cached content for this request, and don't store what comes back either. If you only set

proxy_cache_bypass

, you might still write the response to cache and accidentally serve it to someone else later.

---

Full Configuration Example

Here's the complete, production-ready configuration combining everything above. This goes into

/etc/nginx/conf.d/solvethenetwork.conf

. I've split it into two location blocks — one for static assets with aggressive TTLs, and one for dynamic content with tighter controls:

proxy_cache_path /data/nginx/cache
    levels=1:2
    keys_zone=STATIC_CACHE:50m
    max_size=20g
    inactive=60m
    use_temp_path=off;

map $http_cookie $no_cache_cookie {
    default         0;
    "~*session_id"  1;
    "~*auth_token"  1;
}

map $request_method $no_cache_method {
    default  0;
    POST     1;
    PUT      1;
    DELETE   1;
    PATCH    1;
}

upstream backend_app {
    server 192.168.10.20:8080;
    keepalive 32;
}

server {
    listen 80;
    server_name solvethenetwork.com www.solvethenetwork.com;

    access_log /var/log/nginx/solvethenetwork_access.log combined;
    error_log  /var/log/nginx/solvethenetwork_error.log warn;

    # Static assets — cache aggressively, hide Set-Cookie
    location ~* \.(jpg|jpeg|png|gif|ico|css|js|woff2|woff|ttf|svg|webp)$ {
        proxy_pass          http://backend_app;
        proxy_cache         STATIC_CACHE;
        proxy_cache_valid   200 7d;
        proxy_cache_valid   404 1m;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        proxy_cache_lock    on;
        proxy_cache_bypass  $no_cache_cookie;
        proxy_no_cache      $no_cache_cookie;

        proxy_set_header    Host              $host;
        proxy_set_header    X-Real-IP         $remote_addr;
        proxy_set_header    X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto $scheme;

        proxy_hide_header   Set-Cookie;

        add_header          X-Cache-Status    $upstream_cache_status;
        add_header          Cache-Control     "public, max-age=604800, immutable";
    }

    # Dynamic content — shorter TTL, full bypass logic, tuned buffers
    location / {
        proxy_pass          http://backend_app;
        proxy_cache         STATIC_CACHE;
        proxy_cache_key     "$scheme$host$request_uri";
        proxy_cache_valid   200 302 5m;
        proxy_cache_valid   404     1m;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        proxy_cache_lock    on;
        proxy_cache_lock_timeout 5s;
        proxy_cache_bypass  $no_cache_cookie $no_cache_method;
        proxy_no_cache      $no_cache_cookie $no_cache_method;

        proxy_set_header    Host              $host;
        proxy_set_header    X-Real-IP         $remote_addr;
        proxy_set_header    X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto $scheme;
        proxy_set_header    Connection        "";

        proxy_http_version  1.1;
        proxy_connect_timeout 5s;
        proxy_send_timeout    60s;
        proxy_read_timeout    60s;

        proxy_buffering      on;
        proxy_buffer_size    16k;
        proxy_buffers        8 32k;
        proxy_busy_buffers_size 64k;

        add_header          X-Cache-Status    $upstream_cache_status;
    }
}

Two things worth calling out explicitly in this config. First, the static assets location uses

proxy_hide_header Set-Cookie

. If your backend is accidentally setting session cookies on image or CSS responses — which happens more than you'd think, especially with frameworks that set cookies on every response — those cookies will cause Nginx to skip caching the response entirely. Hiding the header at the proxy layer fixes the behavior without requiring a backend deployment.

Second, the

proxy_http_version 1.1

combined with

proxy_set_header Connection ""

in the dynamic location enables HTTP keepalives on the upstream connection. Without this, Nginx defaults to HTTP/1.0 for upstream requests, which forces a new TCP connection for every proxied request. On a site doing any real traffic, that TCP overhead adds up fast. The

keepalive 32

in the upstream block tells Nginx to keep up to 32 idle connections to the backend per worker process — adjust based on your backend's connection limits and your worker count.

---

Verification Steps

Test the Configuration Syntax

Always run this before reloading. Don't skip it, even for a one-line change:

nginx -t
# nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
# nginx: configuration file /etc/nginx/nginx.conf test is successful

If it passes, reload gracefully. Don't restart — a graceful reload drains existing connections before applying the new config, while a restart drops them:

nginx -s reload

Check Cache Headers with curl

Hit the server twice in quick succession and watch the

X-Cache-Status

header change:

curl -s -I http://solvethenetwork.com/
# X-Cache-Status: MISS

curl -s -I http://solvethenetwork.com/
# X-Cache-Status: HIT

If you're getting MISS on every request, check the upstream response headers with

curl -s -I http://192.168.10.20:8080/

directly from sw-infrarunbook-01. Look for

Cache-Control: no-store

,

Cache-Control: private

, or

Pragma: no-cache

. Any of these will cause Nginx to skip caching regardless of your

proxy_cache_valid

settings.

Inspect the Cache Directory

After warming up a few requests, files should be appearing under

/data/nginx/cache

:

find /data/nginx/cache -type f | wc -l
du -sh /data/nginx/cache

If you want to peek inside a cache file and see what Nginx actually stored, the raw HTTP response is readable with

strings

:

strings $(find /data/nginx/cache -type f | head -1) | head -30

You'll see the HTTP response headers, the cache key, and the expiry timestamp at the top of the file, followed by the response body. This is useful when you're debugging unexpected cache misses and want to confirm that the response was actually written to disk.

Monitor Cache Hit Rate via Access Logs

Add the

$upstream_cache_status

variable to a custom log format so you can track hit rates over time without relying on external metrics:

log_format cache_log '$remote_addr - [$time_local] "$request" '
                     '$status $body_bytes_sent '
                     'cache=$upstream_cache_status rt=$request_time';

access_log /var/log/nginx/solvethenetwork_access.log cache_log;

Then pull the hit rate distribution with a quick awk one-liner against the log:

awk -F'cache=' '{print $2}' /var/log/nginx/solvethenetwork_access.log | \
    awk '{print $1}' | sort | uniq -c | sort -rn

In a well-tuned setup serving a content-heavy site, you should see HIT as the dominant entry. If MISS is leading, your

inactive

timeout might be too short, your TTLs too low, or your cache key too granular (for example, including query strings that vary on every request).

Verify Bypass Behavior

Confirm that authenticated requests are bypassing the cache as expected:

curl -s -I -H "Cookie: session_id=abc123" http://solvethenetwork.com/
# X-Cache-Status: BYPASS

And verify that POST requests aren't being cached either:

curl -s -I -X POST http://solvethenetwork.com/api/data
# X-Cache-Status: BYPASS

---

Common Mistakes

Caching Responses That Set Cookies

This is the most dangerous mistake you can make with a caching proxy, and it's surprisingly easy to do. If your backend sets a

Set-Cookie

header on a response that Nginx caches, every user who receives that cached response will get the exact same cookie that was set for the original requestor. In my experience, this surfaces as session mixing — users suddenly appearing logged in as someone else, or seeing another user's cart. It's the kind of bug that doesn't appear in dev or staging because cache hit rates are low, and then detonates in production under load. Always audit your backend's response headers before enabling caching on a new service. Use

proxy_hide_header Set-Cookie

on any location block where cookies have no business appearing.

Under-Sizing the Keys Zone

If your

keys_zone

shared memory fills up, Nginx's cache manager starts evicting metadata entries to make room. This means cache misses start spiking even though the content files are still sitting on disk — Nginx just lost its index. You won't get an obvious error message. You'll just see your hit rate quietly collapse. If you're caching a site with many unique URLs, increase the keys_zone size. The cost is minimal — memory allocated for zones is reserved but lightweight compared to the actual cache content.

Not Setting proxy_cache_lock

Without

proxy_cache_lock on

, a thundering herd scenario becomes trivial to trigger. A popular cached page expires. In the milliseconds before the first request repopulates the cache, fifty concurrent requests arrive, all get a MISS, and all of them fire upstream simultaneously. Under serious load, this can spike backend CPU dramatically and potentially knock over an application server that was otherwise handling steady-state traffic fine. With

proxy_cache_lock on

, only the first request goes upstream while the rest wait. Set

proxy_cache_lock_timeout

to something slightly above your backend's typical p95 response time so waiting requests don't bail out too early.

Ignoring Cache-Control Headers from the Backend

Nginx respects upstream

Cache-Control

and

Expires

headers by default. If your backend sends

Cache-Control: no-store

or

Cache-Control: private

, Nginx will not cache the response — even if you have

proxy_cache_valid

explicitly configured. In my experience, this catches engineers off guard after they've carefully written all the right Nginx directives and nothing is getting cached. The fix is either to correct the backend headers (preferred), or to add

proxy_ignore_headers Cache-Control Expires

in the location block if you explicitly want Nginx to override them. If you do override, document it — it's a footgun that will confuse whoever maintains this config six months from now.

Forgetting use_temp_path=off

Without this directive, Nginx writes incoming backend responses to a temporary directory first, then renames the file to its final location in the cache path. If both paths are on the same filesystem, the rename is essentially free. But if you have a dedicated mount for

/data/nginx/cache

while your temp path is still on the root filesystem, every cache write becomes a cross-device file copy — double the disk I/O for every single MISS. Always set

use_temp_path=off

explicitly so you're not surprised by this behavior if someone ever remounts the cache path.

No Plan for Cache Warm-Up After Restarts

This isn't a config mistake, but it's an operational one that costs teams dearly. A cold cache on a high-traffic site means your backend takes 100% of request load immediately after a deployment or system restart. If your backend was already running close to capacity under normal steady-state conditions — with the cache absorbing the majority of reads — a cold restart can cause cascading failures before the cache repopulates. Either pre-warm the cache by crawling your sitemap after deployment, or ensure

proxy_cache_use_stale updating

is configured so that when a cached item needs refreshing, stale content is served to waiting clients while one request goes upstream. Don't leave your backend exposed to cold-start load spikes without a plan.