net/http: allow customizing proxy CONNECT url

[liuxingbaoyu]

Update May 5 2021: The current proposed API is in #45460 (comment) - rsc

---

What version of Go are you using (go version)?

go version go1.16 windows/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

set GO111MODULE=auto
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\流星暴雨\AppData\Local\go-build
set GOENV=C:\Users\流星暴雨\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=E:\Eternal\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=E:\Eternal\go;
set GOPRIVATE=
set GOPROXY=https://goproxy.io,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.16
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\ 流星暴雨\AppData\Local\Temp\go-build1194816388=/tmp/go-build -gno-record-gcc-switches

What did you do?

I want to use a proxy to access HTTPS websites and customize the backend IP address

What did you expect to see?

Is there some way to customize the URL of the CONNECT request

What did you see instead?

Both CONNECT request and SSL handshake use cm.targetAddr

go/src/net/http/transport.go

Lines 1667 to 1672 in 2ebe77a

	connectReq := &Request{
	Method: "CONNECT",
	URL: &url.URL{Opaque: cm.targetAddr},
	Host: cm.targetAddr,
	Header: hdr,
	}

go/src/net/http/transport.go

Lines 1725 to 1729 in 2ebe77a

	if cm.proxyURL != nil && cm.targetScheme == "https" {
	if err := pconn.addTLS(ctx, cm.tlsHost(), trace); err != nil {
	return nil, err
	}
	}

go/src/net/http/transport.go

Lines 1838 to 1846 in 2ebe77a

	// tlsHost returns the host name to match against the peer's
	// TLS certificate.
	func (cm *connectMethod) tlsHost() string {
	h := cm.targetAddr
	if hasPort(h) {
	h = h[:strings.LastIndex(h, ":")]
	}
	return h
	}

[seankhliao]

Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.

For questions please refer to https://github.com/golang/go/wiki/Questions

[liuxingbaoyu]

@seankhliao Oh no, I'm sorry that I didn't express it clearly. I read the relevant code and did not find a way to customize the CONNECT request. Perhaps this requires a new API.

[seankhliao]

what exactly do you want to do? change the address the of the proxy while keeping the hostname?

[liuxingbaoyu]

When an HTTPS request is sent through an HTTP proxy, it will first send a CONNECT request to establish a TCP connection, and then it will start a handshake with the SSL of the website. If a website has multiple resolution addresses, such as using a CDN, you can use the specified backend IP to access the website by specifying the CONNECT URL. I hope there is an API that can customize the CONNECT URL or customize the entire CONNECT request.
Similar to #13290

[seankhliao]

Right, different values for CONNECT xxx (for connecting) and Host: yyy (for TLS)

[seankhliao]

cc @bradfitz @neild @fraenkel

[rsc]

The code that sets up the CONNECT does:

connectReq := &Request{
	Method: "CONNECT",
	URL:    &url.URL{Opaque: cm.targetAddr},
	Host:   cm.targetAddr,
	Header: hdr,
}

If hdr["Host"] is set, then it sounds like we should use it instead of cm.targetAddr to set up connectReq.Host. That is, add:

if host, ok := hdr["Host"]; ok {
    if len(host) >= 1 {
        connectReq.Host = host[0]
    } else {
        connectReq.Host = ""
    }
}

Does anyone have any objections to this?

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[fraenkel]

Sorry but I am confused on what we are trying to solve.

We have a request to some target site, www.target.org, and we are going through some Proxy. The actual request has a URL/host of www.target.org. It really doesn't matter what the host header is. The Request will ignore the Host header and generate its own based, https://github.com/golang/go/blob/master/src/net/http/request.go#L570

A proxy is required to ignore the host header if it receives an absolute uri, https://tools.ietf.org/html/rfc7230#section-5.4

When a proxy receives a request with an absolute-form of
   request-target, the proxy MUST ignore the received Host header field
   (if any) and instead replace it with the host information of the
   request-target.