Go http.Client: unexpected EOF and Connection Reuse Leaks
A body closed without being read cannot be reused, and descriptors pile up. Idle timeouts shorter than the upstream's, MaxIdleConnsPerHost and safe retries.

Introduction & Quick Summary
Symptom. A Go service that calls an upstream API shows Get "https://api.example.com/v1/items": unexpected EOF or http: server closed idle connection a few times an hour, always on the first request after a quiet period. File descriptors climb towards the limit and dial tcp: lookup api.example.com: no such host appears once the process runs out. Under load, every request opens a new TCP connection even though keep-alive is on.
Cause. Two habits and one default. A resp.Body closed without being fully read leaves the transport unable to reuse its connection, because the unread remainder makes the stream state unknown, so the connection is dropped and, if the body is never closed at all, the file descriptor leaks. Separately, an upstream or load balancer closes idle connections after its own timeout, and the client's IdleConnTimeout (90 seconds by default) is longer, so the first reuse of a connection the server has already closed fails — the transport retries it only when the request is replayable. And MaxIdleConnsPerHost defaults to 2, so a client with high concurrency keeps two connections warm and dials the rest.
Fix. Always io.Copy(io.Discard, resp.Body) before Close() (or read to EOF), tune a cloned Transport with MaxIdleConnsPerHost, IdleConnTimeout, and ResponseHeaderTimeout set below the upstream's idle timeout, and treat one retry of an idempotent request as normal operation.
Core Mechanics, Signatures & Reproducible Setup
Connection reuse depends on the body being consumed to its end. Everything else in this article follows from that.
func get(ctx context.Context, client *http.Client, url string) ([]byte, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
// drain first: without this the connection cannot be reused
body, err := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
if err != nil {
return nil, err
}
return body, nil
}io.ReadAll reads to EOF, which is what the transport needs. The failure mode appears when the response is large and the caller stops early — a JSON decoder that stops after the first object, an if resp.StatusCode != 200 { return } that returns before touching the body.
// the leak: a non-200 response returned without reading anything
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("upstream %d", resp.StatusCode) // body never drained
}
// the fix
if resp.StatusCode != http.StatusOK {
io.Copy(io.Discard, io.LimitReader(resp.Body, 64<<10))
return fmt.Errorf("upstream %d", resp.StatusCode)
}A client with sane limits, built once at startup:
func newClient() *http.Client {
t := http.DefaultTransport.(*http.Transport).Clone()
t.MaxIdleConns = 100
t.MaxIdleConnsPerHost = 20 // default is 2
t.IdleConnTimeout = 30 * time.Second // shorter than the upstream's 60s
t.ResponseHeaderTimeout = 10 * time.Second
t.TLSHandshakeTimeout = 5 * time.Second
return &http.Client{Transport: t, Timeout: 30 * time.Second}
}Reproduce the idle-connection failure with a server that closes after two seconds and a client that waits three, then reuses. Go returns unexpected EOF on the first attempt and succeeds on the automatic retry for a GET; a POST with a body fails outright, because the transport will not replay a request whose body it has already consumed.
Key Comparison Table
| Setting | Default | Failure when left alone |
|---|---|---|
MaxIdleConnsPerHost | 2 | dial churn under concurrency |
IdleConnTimeout | 90s | reuse of connections the server already closed |
MaxConnsPerHost | 0 (unlimited) | no backpressure toward a struggling upstream |
ResponseHeaderTimeout | 0 (none) | goroutines stuck on a silent server |
Client.Timeout | 0 (none) | the whole request, body included, never expires |
DisableKeepAlives | false | setting it true recreates every connection |
| Closing pattern | Connection reused | Descriptor freed |
|---|---|---|
defer resp.Body.Close() only | no, if body unread | yes |
io.Copy(io.Discard, body) then Close | yes | yes |
| Neither | no | no, leaks |
ReadAll then Close | yes | yes |
Close on a 4xx without reading | no | yes |
Practical Edge Cases, Error Fixes & Deep Dive Code Examples
1. unexpected EOF on a reused connection
The upstream closed the socket while the client still believed it was idle. Since Go 1.6 the transport retries when the request is idempotent and no response bytes arrived, so a GET usually succeeds transparently and the error surfaces only on POST, PATCH, or a request with a GetBody that is nil.
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(payload))
req.GetBody = func() (io.ReadCloser, error) {
return io.NopCloser(bytes.NewReader(payload)), nil // enables safe retry
}With GetBody set, the transport may replay the request on a fresh connection. Without it, the error is final, and the correct client behaviour is to retry at the application layer with an idempotency key.
2. Leaked descriptors from early returns
Each unclosed body holds a connection and a file descriptor. A service doing 50 requests per second with a 5% early-return path leaks about 2.5 descriptors per second, which reaches ulimit -n in a couple of hours.
# confirm the leak before changing code
curl -s localhost:6060/debug/pprof/goroutine?debug=2 | grep -c "net/http.(*persistConn)"
lsof -p $(pgrep -f myservice) | grep -c TCPA rising count of persistConn goroutines that never falls is the signature. Goroutine profiles of a http.Client leak almost always point at a body that was never closed.
3. Client.Timeout also bounds the body read
Timeout covers dial, request write, response headers, and reading the body. A client with Timeout: 30 * time.Second reading a 2 GB download fails at thirty seconds even though the connection is healthy. For streaming, leave Timeout at zero and set ResponseHeaderTimeout on the transport plus a per-request context deadline.
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
defer cancel()
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, bigURL, nil)4. A global client versus a client per request
http.Client is safe for concurrent use and holds the connection pool. Creating one per request discards the pool each time, so every call dials and handshakes.
// wrong
func fetch(url string) (*http.Response, error) { return (&http.Client{}).Get(url) }
// right
var shared = newClient()The same applies to http.DefaultClient, whose DefaultTransport has the two-idle-connection default and no timeouts at all — the reason http.Get is fine for a script and wrong for a service.
5. HTTP/2 and ForceAttemptHTTP2
A custom Transport built from scratch rather than cloned silently loses HTTP/2, because ForceAttemptHTTP2 is only true when the transport has the standard dialer and TLS config. The symptom is a client that negotiated h2 with the default transport and falls back to HTTP/1.1 after the custom transport is introduced.
t.ForceAttemptHTTP2 = true6. Reading the connection state
httptrace answers the question the error message cannot: was this request reused or dialled?
trace := &httptrace.ClientTrace{
GotConn: func(info httptrace.GotConnInfo) {
log.Printf("reused=%v idle=%v remote=%v", info.Reused, info.WasIdle, info.Conn.RemoteAddr())
},
}
req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))A reused=false on every request in steady state means the body is not being drained somewhere upstream of this call.
Common Compilation/Runtime Mistakes & Version Gotchas
defer resp.Body.Close()inside a loop. The defer runs at function exit, so N responses stay open until the loop ends. Wrap each iteration in a function.- Closing without draining. The connection is discarded rather than reused; it is a performance bug that looks like an upstream latency problem.
MaxIdleConnsPerHostleft at 2. Under 100 concurrent requests, 98 connections are created and thrown away per burst.IdleConnTimeoutlonger than the upstream's. Predictableunexpected EOFon the first request after idleness.DisableKeepAlives: true"to fix" EOF errors. It multiplies handshakes and removes the retry that madeGETreliable.- Custom
Transportbuilt literally. Loses HTTP/2, proxy from environment, and sensible dial timeouts. - Retrying a
POSTwithout an idempotency key. A replayed write creates duplicates; the safe retry is at the application layer with a key the server deduplicates on.
Related tools. Turn a curl -v transcript of the upstream into a Go client with cURL to code, inspect the JSON body of a failed response with the JSON formatter, and confirm two responses differ by more than formatting with the text diff.
Try matching browser tools
All toolsSmall utilities that fit Go work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Go guides
All Go guidesGo context.DeadlineExceeded vs Canceled and Goroutine Leaks
A deadline is retryable and a cancellation is not, and both arrive wrapped. The errors.Is checks, the four handler timeouts, and the goroutine dump that names the leak.
Go Module Errors: Checksum Mismatch, 404s, and Proxy Failures
A checksum mismatch is not a cache bug. Separate the proxy, the sum database, and go.sum, then fix the layer the message names instead of editing the hash.
Go iota: Enumerations, Bitmask Flags, and Constant Expressions
How iota advances per const line, typed enums with gaps, bit flags with 1 << iota, and the unit scale pattern that shifts with one blank line.
Go JSON: Marshaling, Unmarshaling, Struct Tags, and Custom Parsers
Struct tags, omitempty that skips false and zero, custom marshalers without recursion, RawMessage for tagged unions, and float64 precision loss.
More in Go
All Go guidesThe guides filed next to this one in Go, in library order.