net/http: client closes persistent connections when context deadline is exceeded without reuse

[akhilesh-godi]

What version of Go are you using (go version)?

$ go version
1.14.4

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/godia/Library/Caches/go-build"
GOENV="/Users/godia/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/godia/dev"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/r0/ggt8ftqn467cckf8r_y58vqr0000gp/T/go-build603942499=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Ran into a scenario while using the http client, when there is a surge in the number of requests and in certain requests when context deadline is exceeded at the client side.

What did you expect to see?

Persistent connection remains intact when not explicitly closed due to request context timeouts.

What did you see instead?

Persistent connections should be reused. But a lot of new dials happen due to closing of persistent connections which could have been potentially avoided.

Relevant code in question: https://github.com/golang/go/blob/master/src/net/http/transport.go#L2219

[mknyszek]

CC @bradfitz @rsc @empijei via https://dev.golang.org/owners

[networkimprov]

@akhilesh-godi have you tried this in Go 1.16 or 1.15? Support for 1.14 has ended.

cc @neild @fraenkel

[fraenkel]

This is the correct behavior.
Lets say you sent a request and the deadline hits. The response is never read, now what do you do?
Unless the request/response is completed, it is more correct to just throw out the connection.

If you have a specific case in mind where it can safely be used, you need to provide an example so we can examine further.