Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

nginx 502 Bad Gateway: upstream connection refused

The log line names the address and errno, which is the diagnosis. Fix the host, the start-up order, the buffers, or the timeout that actually fired.

~5 min readLinux
#nginx#Docker#Reverse Proxy#Troubleshooting
A proxy gate with a sealed upstream door and a refused arrow, headline BAD GATEWAY

Introduction & Quick Summary

Symptom. nginx answers with its own error page, not the application's:

502 Bad Gateway

and the error log names the upstream precisely:

code
[error] 31#31: *7 connect() failed (111: Connection refused) while connecting to upstream [error] 31#31: *9 upstream prematurely closed connection while reading response header

Cause. nginx is a reverse proxy here: it accepted the client connection and could not get a valid response from the upstream server. 111: Connection refused means nothing listened at the address nginx tried. upstream prematurely closed connection means something listened and closed without a complete header, often an application crash or an upstream proxy timeout. no live upstreams means every backend failed its passive health checks. upstream sent too big header is a buffer limit.

Fix. Read the two values in the log line: the address nginx dialled and the errno. In Docker Compose the usual mistake is proxy_pass http://127.0.0.1:3000, which inside the nginx container points at nginx itself, not at the app container:

nginx
location / { proxy_pass http://app:3000; # service name on the compose network proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; }

Core Mechanics, Signatures & Reproducible Setup

proxy_pass resolves its host once at start-up when the target is a static name in an upstream block, and per request when it is a variable with a resolver. The connection is opened with a timeout (proxy_connect_timeout, default 60s), then the request is sent and the response header read with proxy_read_timeout. Each phase has its own error string, which is why the log line is more useful than the status code.

Reproduce all four common forms with a minimal compose file:

services: proxy: image: nginx:1.27-alpine ports: ["8080:80"] volumes: ["./default.conf:/etc/nginx/conf.d/default.conf:ro"] app: image: hashicorp/http-echo:1.0 command: ["-listen=:3000", "-text=ok"]
nginx
server { listen 80; location / { proxy_pass http://app:3000; } location /dead { proxy_pass http://127.0.0.1:9999; } }

curl -i localhost:8080/ returns ok; localhost:8080/dead returns 502 with connect() failed (111: Connection refused) in docker compose logs proxy.

Key Comparison Table

Match the log line to the cause:

Error log textCauseFix
connect() failed (111: Connection refused)Nothing listening at that host and portCorrect the host, port, or start the upstream
connect() failed (110: Connection timed out)Packets dropped, wrong network, firewallCheck routes, security groups, service exposure
upstream prematurely closed connectionUpstream process died or timed out firstRead upstream logs; raise its timeout
no live upstreams while connectingAll backends marked failedFix one backend, or adjust max_fails
upstream sent too big headerResponse header larger than bufferRaise proxy_buffer_size and buffers
recv() failed (104: Connection reset by peer)Kernel-level reset, often a half-closed socketCheck keepalive settings on both sides
SSL_do_handshake() failedTLS mismatch to the upstreamUse the right scheme and SNI
Client seesMeaningWhere to look first
502 for every requestConfig or upstream downnginx error log, then upstream process
502 for some requestsOne backend of several is badupstream block, health checks
504 instead of 502Upstream accepted but did not answer in timeproxy_read_timeout, app latency
502 only after a deployOld container stopped before new one was readyDepends_on with healthcheck, rolling start

Practical Edge Cases, Error Fixes & Deep Dive Code Examples

1. Docker networking mistakes

Three host names mean different things inside a container: 127.0.0.1 is the container itself, host.docker.internal is the Docker host (Desktop only), and a Compose service name resolves through Docker DNS. A proxy_pass http://localhost:3000 in the nginx container always fails with 111 because nginx has nothing on port 3000.

bash
docker compose exec proxy getent hosts app # must print an IP docker compose exec proxy wget -qO- http://app:3000/ # must print ok

If getent fails, the containers are not on the same network; check docker compose config for a shared networks key.

2. Start-up order and health checks

depends_on waits for the container to start, not for the process to listen. nginx starts first, caches a failed resolution, and returns 502 until it is reloaded. Two fixes, preferably both:

services: app: healthcheck: test: ["CMD", "wget", "-qO-", "http://localhost:3000/health"] interval: 5s retries: 10 proxy: depends_on: app: condition: service_healthy

For a dynamic upstream (Kubernetes, ECS, Compose scale), use a resolver so nginx re-resolves instead of caching the first IP:

nginx
resolver 127.0.0.11 valid=10s ipv6=off; set $upstream http://app:3000; proxy_pass $upstream;

3. Buffer and header limits

A large Set-Cookie or a JWT in a header can exceed nginx's default 4 KB header buffer, producing upstream sent too big header and a 502 that appears only for logged-in users:

nginx
proxy_buffer_size 16k; proxy_buffers 4 32k; proxy_busy_buffers_size 64k; large_client_header_buffers 4 16k;

The same symptom with a 400 instead of 502 usually means the client header was too large, which large_client_header_buffers covers.

4. Timeouts that look like crashes

An application that takes longer than proxy_read_timeout (60s by default) is closed by nginx, and the app logs a broken pipe. The client sees 504, and the nginx log shows upstream timed out (110: Connection timed out) while reading response header. For long-running endpoints, align three values: the app's own timeout, proxy_read_timeout, and the load balancer in front of nginx. Raising only one moves the failure to the next hop.

5. PHP-FPM and 502 after a restart

With fastcgi_pass unix:/run/php/php8.3-fpm.sock, a 502 usually means the socket path in the pool config differs from the one nginx uses, or the pool exhausted its children and queued connections until request_terminate_timeout. Check php-fpm logs and ss -xl | grep php for the actual socket path, and raise pm.max_children when the log shows server reached pm.max_children.

6. Verifying from the right place

curl localhost:3000 on the host proves nothing about what nginx can reach. Run the check inside the proxy's network namespace, or at least from another container on the same network. A 502 that disappears when tested from the host is a network-scope problem every time.

Common Compilation/Runtime Mistakes & Version Gotchas

  • Assuming 502 means nginx is broken. nginx is reporting an upstream failure; the upstream is the subject.
  • Using localhost in proxy_pass across containers. It resolves to the nginx container. Use the service name or the host's address.
  • Reloading nginx after changing DNS. A static proxy_pass host is resolved at start-up; a reload is required, and a resolver is required for changes in IP.
  • Setting proxy_set_header Connection "" without keepalive upstreams. The client connection closes per request, which is slower but not broken; the combination with an upstream block without keepalive is the usual misconfiguration.
  • Editing the default nginx.conf instead of a conf.d file. The image includes /etc/nginx/conf.d/default.conf with a server block on port 80 that shadows yours.
  • Blaming nginx 1.27 for a new 502. Version changes rarely cause this; a changed upstream port or a new health check endpoint is far more common.

Confirm the status semantics with the HTTP status code reference, validate the compose file with the Docker Compose formatter, and compute the client network range for allow lists with the CIDR calculator.

Reference: nginx proxy_pass and ngx_http_upstream_module.

Official references

primary documentation

The Linux documentation that defines the behaviour this guide explains.

Try matching browser tools

All tools

Small utilities that fit Linux work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Linux guides

More in Linux

All Linux guides

The guides filed next to this one in Linux, in library order.