Prerequisites
Before we get into the weeds, make sure you have the following in place. Skipping any of these is the number one reason people end up spending two hours debugging what should be a 20-minute setup.
- A running Kubernetes cluster (1.24 or newer). This guide was validated on both k3s and kubeadm clusters.
- kubectl configured with cluster-admin privileges
- Helm 3 installed on your local machine
- A domain you control — we'll use solvethenetwork.com throughout this guide
- DNS A records pointing your subdomain to your ingress controller's external IP before you attempt certificate issuance
- Ports 80 and 443 open inbound on your load balancer or node-level firewall
One thing I see overlooked constantly: DNS propagation. You can have a perfectly valid cert-manager configuration and still get ACME HTTP-01 challenge failures because the DNS record hasn't propagated yet. Give it time — or switch to DNS-01 challenges if you're working in an environment where you can't wait. We'll cover that option in the common mistakes section.
Step-by-Step Setup
Step 1: Install the NGINX Ingress Controller
We'll use the community NGINX ingress controller via Helm. This is the one maintained at
kubernetes/ingress-nginx, not the NGINX Inc. version — they differ in annotation support and behavior, so make sure you're pulling from the right chart repo.
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--create-namespace \
--set controller.replicaCount=2 \
--set controller.nodeSelector."kubernetes\.io/os"=linux \
--set controller.service.type=LoadBalancer
After installation, pull the external IP assigned to the LoadBalancer service. On bare metal with MetalLB (like the setup on
sw-infrarunbook-01), this will be a static IP from your configured address pool. On cloud providers it'll be dynamically assigned.
kubectl get svc -n ingress-nginx ingress-nginx-controller
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
ingress-nginx-controller LoadBalancer 10.96.14.22 192.168.10.50 80:31080/TCP,443:31443/TCP
That
192.168.10.50is what your DNS A record needs to point to. Create an A record for
app.solvethenetwork.comtargeting that IP before you move on to certificate provisioning.
Step 2: Install cert-manager
cert-manager is the de facto standard for automated TLS certificate management in Kubernetes. It speaks ACME protocol fluently, handles Let's Encrypt interactions, and watches your Ingress objects to automatically provision and renew certificates without you touching anything.
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.14.4 \
--set installCRDs=true
The
installCRDs=trueflag is non-negotiable. Without it, none of the cert-manager custom resources —
ClusterIssuer,
Certificate,
CertificateRequest,
Order,
Challenge— will exist in your cluster. I've watched people install cert-manager and immediately wonder why their YAML refuses to apply. Nine times out of ten, it's missing CRDs.
Wait for all three cert-manager deployments to fully roll out before continuing. The webhook in particular needs to be healthy before you can create any cert-manager resources:
kubectl rollout status deployment/cert-manager -n cert-manager
kubectl rollout status deployment/cert-manager-webhook -n cert-manager
kubectl rollout status deployment/cert-manager-cainjector -n cert-manager
Step 3: Create a ClusterIssuer
A
ClusterIssueris a cluster-scoped resource that tells cert-manager how to obtain certificates. It's the cluster-wide counterpart to a namespace-scoped
Issuer. We're using
ClusterIssuerbecause it can serve multiple namespaces, which is almost always what you want in practice.
Start with the staging issuer. Let's Encrypt staging has much higher rate limits, and it lets you verify your entire provisioning pipeline before touching your production quota. In my experience, skipping staging and going straight to production is how teams hit rate limit walls at the worst possible time.
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: infrarunbook-admin@solvethenetwork.com
privateKeySecretRef:
name: letsencrypt-staging-key
solvers:
- http01:
ingress:
class: nginx
kubectl apply -f clusterissuer-staging.yaml
Once staging validates your setup end to end, create the production issuer:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: infrarunbook-admin@solvethenetwork.com
privateKeySecretRef:
name: letsencrypt-prod-key
solvers:
- http01:
ingress:
class: nginx
kubectl apply -f clusterissuer-prod.yaml
Step 4: Deploy a Test Application
We need something to sit behind the ingress. Here's a minimal deployment and service that's good enough to verify the full stack:
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: web-app
template:
metadata:
labels:
app: web-app
spec:
containers:
- name: nginx
image: nginx:1.25-alpine
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: web-app-svc
namespace: default
spec:
selector:
app: web-app
ports:
- protocol: TCP
port: 80
targetPort: 80
kubectl apply -f web-app.yaml
Step 5: Create the Ingress with TLS
This is where it all comes together. The annotation
cert-manager.io/cluster-issueris the trigger — it tells cert-manager to watch this Ingress resource and automatically provision a certificate using the named ClusterIssuer. The
tlsblock declares which secret should hold the resulting certificate and which hostnames it covers. The ingress controller reads that same secret to terminate TLS connections.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web-app-ingress
namespace: default
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- app.solvethenetwork.com
secretName: web-app-tls
rules:
- host: app.solvethenetwork.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web-app-svc
port:
number: 80
kubectl apply -f web-app-ingress.yaml
The moment you apply this, cert-manager starts the certificate request process. It creates a
Certificateobject, which spawns an
Order, which spawns a
Challenge. The challenge mechanism temporarily creates an ingress rule that answers Let's Encrypt's HTTP-01 verification request on port 80. Once verification succeeds, the signed certificate lands in the
web-app-tlssecret and the ingress controller picks it up automatically.
Full Configuration Example
Here's a complete, production-ready manifest from a working bare-metal cluster using MetalLB with address pool
192.168.10.50–192.168.10.60. Everything is namespace-isolated under
web-production, which is how I prefer to structure application workloads on shared clusters.
# Namespace
apiVersion: v1
kind: Namespace
metadata:
name: web-production
---
# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app
namespace: web-production
spec:
replicas: 2
selector:
matchLabels:
app: web-app
template:
metadata:
labels:
app: web-app
spec:
containers:
- name: nginx
image: nginx:1.25-alpine
ports:
- containerPort: 80
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 200m
memory: 128Mi
---
# Service
apiVersion: v1
kind: Service
metadata:
name: web-app-svc
namespace: web-production
spec:
selector:
app: web-app
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
---
# Ingress with TLS
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web-app-ingress
namespace: web-production
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "10m"
nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
nginx.ingress.kubernetes.io/proxy-send-timeout: "60"
spec:
ingressClassName: nginx
tls:
- hosts:
- app.solvethenetwork.com
secretName: web-app-tls
rules:
- host: app.solvethenetwork.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web-app-svc
port:
number: 80
The
ClusterIssueris cluster-scoped and should be applied separately. It doesn't belong inside a namespace-specific manifest file — I keep it in a dedicated
cluster-issuers/directory that gets applied once during cluster bootstrapping.
Verification Steps
Once everything is applied, here's how to confirm certificate provisioning worked and that the ingress is actually serving HTTPS traffic correctly. Don't skip these — a certificate in
Pendingstate won't tell you it's broken until a client hits it.
Check the Certificate Status
kubectl get certificate -n web-production
NAME READY SECRET AGE
web-app-tls True web-app-tls 3m12s
READY: Trueis the goal. If it's stuck on
False, describe the certificate object to see the condition messages:
kubectl describe certificate web-app-tls -n web-production
Trace the Challenge Chain
cert-manager creates a chain of objects during provisioning. You can watch each stage:
kubectl get certificaterequest -n web-production
kubectl get order -n web-production
kubectl get challenge -n web-production
A challenge will progress through
pendingto
validand then disappear — cert-manager cleans them up on success. If a challenge stays in
pendinglonger than a couple of minutes, that's your signal to check DNS resolution and port 80 accessibility from the public internet.
Confirm the TLS Secret Exists
kubectl get secret web-app-tls -n web-production
NAME TYPE DATA AGE
web-app-tls kubernetes.io/tls 2 3m8s
The type must be
kubernetes.io/tlsand the DATA column should show 2 — that's
tls.crtand
tls.key. If either key is missing, the ingress controller can't serve TLS and will fall back to its default self-signed certificate.
Test HTTPS End to End
curl -v https://app.solvethenetwork.com 2>&1 | grep -A5 "Server certificate"
* Server certificate:
* subject: CN=app.solvethenetwork.com
* issuer: C=US, O=Let's Encrypt, CN=R11
* SSL certificate verify ok.
If you're still on staging, the verify will fail because Let's Encrypt staging certificates aren't trusted by your OS certificate store. Use
curl -kduring staging tests to bypass verification. Once you've confirmed the provisioning pipeline works, flip the annotation to
letsencrypt-prod, delete the existing secret, and let cert-manager reissue.
Verify HTTP Redirects to HTTPS
curl -I http://app.solvethenetwork.com
HTTP/1.1 308 Permanent Redirect
Location: https://app.solvethenetwork.com
A 308 confirms the ssl-redirect annotations are working. If you get a 200 back over plain HTTP, the annotations aren't taking effect — check that you're running a recent enough version of the ingress controller and that the annotation names are spelled correctly.
Common Mistakes
Using the Old Ingress Class Annotation
Before Kubernetes 1.18, you set the ingress class via an annotation:
kubernetes.io/ingress.class: nginx. From 1.18 onwards, the correct approach is the
spec.ingressClassNamefield. On newer clusters, the old annotation may be silently ignored, meaning your ingress controller never picks up the resource. I've seen teams spend an entire afternoon on this — the ingress exists, looks correct, and does absolutely nothing.
Port 80 Blocked at the Firewall
HTTP-01 challenges require port 80 to be reachable from the Let's Encrypt validation servers. If your cluster sits behind a firewall that only allows inbound 443, the ACME challenge will sit in
pendingindefinitely. Either open port 80, or switch to DNS-01 challenges. DNS-01 validates ownership via a
_acme-challengeTXT record instead of an HTTP endpoint, so it doesn't require any inbound HTTP access. It does require API credentials for your DNS provider, but for environments with strict inbound firewall rules it's worth the extra setup.
Mismatched Hostnames Between TLS and Rules
The hostname in
spec.tls[].hosts[]must exactly match the hostname in
spec.rules[].host. If they're different — even by a trailing dot or a capitalization difference — the ingress controller will serve its default self-signed certificate instead of the one you provisioned. Clients will get a certificate name mismatch error that looks nothing like an ingress misconfiguration. This is one of those bugs that's embarrassingly easy to introduce when copy-pasting configs between environments.
Forgetting to Delete the Staging Secret Before Switching to Production
After testing with staging, the
web-app-tlssecret already contains a staging certificate. When you update the annotation to
letsencrypt-prod, cert-manager sees a valid-looking (non-expired) secret and won't replace it automatically. You have to delete it manually so cert-manager detects the absence and kicks off a new order:
kubectl delete secret web-app-tls -n web-production
cert-manager will notice the missing secret within a minute and start the production certificate request. No need to delete or re-apply the Ingress.
Burning Through Let's Encrypt Rate Limits
Let's Encrypt production enforces 50 certificates per registered domain per week, with a 5 duplicate certificate cap. In my experience, teams running dev, staging, and production environments on subdomains of the same root domain hit this faster than expected — especially early on when the setup is being iterated. Use the staging issuer during setup, use unique subdomains per environment, and consider wildcard certificates via DNS-01 if you're managing many subdomains. A wildcard for
*.solvethenetwork.comcounts as one certificate regardless of how many subdomains it covers.
Applying cert-manager Resources Before the Webhook Is Ready
If you apply a
ClusterIssuerimmediately after the Helm install completes, you may hit this:
Error from server (InternalError): error when creating "clusterissuer.yaml":
Internal error occurred: failed calling webhook "webhook.cert-manager.io":
the server is currently unable to handle the request
The cert-manager webhook deployment needs a few seconds to become healthy and register itself with the API server. The
kubectl rollout statuscommands in Step 2 block until the webhook is ready — don't skip them, and don't try to parallelize the install and resource creation steps.
One pattern I use consistently in production: apply the Ingress without the TLS block first, confirm traffic routes correctly over HTTP, then add the TLS configuration as a second apply. It's much easier to isolate routing problems from certificate problems when you tackle them in sequence.
Getting Kubernetes Ingress with TLS right the first time is genuinely satisfying. Once cert-manager is running and your first certificate flips to
READY: True, the automated renewal cycle takes over — Let's Encrypt certificates renew 30 days before expiry — and you can stop thinking about certificate management entirely. That's the whole point.
