net/http/httputil: ReverseProxy closing upgraded tcp connection

[matejkramny]

Go version

go version go1.23.2 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/m/.cache/go-build'
GOENV='/home/m/.config/go/env'
GOEXE=''
GOEXPERIMENT='nocoverageredesign'
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/m/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/m/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.23.2'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/m/.config/go/telemetry'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/m/go-proxy-bug/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1314557167=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Impacted by bug in rancher-desktop: rancher-sandbox/rancher-desktop#3239

Rancher-desktop uses net/http/httputil.ReverseProxy to proxy requests from the docker cli on linux to another WSL container via unix sockets.

The Docker API uses a perhaps non-standard HTTP/1.1 websocket request to stream input/output from the containers (docker documentation: https://docs.docker.com/reference/api/engine/version/v1.47/#tag/Container/operation/ContainerAttach).

The bug is that ReverseProxy is closing the connection when the client closes the outbound socket.

I was able to reproduce it, repo: https://github.com/matejkramny/go-proxy-bug

The long and short of the bug reproduction is:

• A websocket server
• Proxy using httputil.ReverseProxy
• Client that disconnects the writing side of the tcp connection

Demonstrated here:
https://github.com/matejkramny/go-proxy-bug/blob/main/client/client_raw.go#L84

net.TCPConn has a CloseWrite function that simulates what the docker client is doing to the underlying connection

I couldn't find much about the Upgrade: tcp header but it's not complying with the websocket standard (RFC: https://datatracker.ietf.org/doc/html/rfc6455). Maybe what Docker is doing seems like non-standard behaviour and abusing the http protocol in some ways which may be unsupported by the ReverseProxy?

What did you see happen?

ReverseProxy is closing the entire connection when a client closes the output.

The server can still write but the client is unable to receive from the server. The proxy might be closing the client connection by mistake because the client closed one of it's streams.

What did you expect to see?

The proxy should still be able to forward data from the server even when the client closed one of the streams.

[gabyhelp]

Related Issues

• net/http/httputil: ReverseProxy doesn't support TCP half-close when HTTP is upgraded #35892
• net/http/httputil: ReverseProxy - websocket connections cannot be canceled #35559 (closed)
• net/http/httputil: ReverseProxy does not pass previous headers when upgrade connection #29407 (closed)
• x/net/http2: closing a requestBody in the ServerHTTP method drops conn-level flow control #28634 (closed)
• x/net/http2: transport race in closing stream while processing DATA frame, kills connection. #47076
• net/http/httputil: ReverseProxy doesn't handle connection/stream reset properly #34413
• x/net,net/http/httputil: ReverseProxy attempts to log each http2 stream cancellation, causing concurrent streams to grow unbounded due to logger buildup #53257
• net/http/httputil: ReverseProxy fails to proxy h2c #33452 (closed)
• net/http/httputil: Grpc over Go Proxy brings always connection error (help wanted) #29928 (closed)

Related Code Changes

• net/http/httputil: make ReverseProxy automatically proxy WebSocket requests

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[seankhliao]

Duplicate of #35892