net/http: NewRequest and NewRequestWithContext do not ensure a valid path

[rustysys-dev]

Go version

go version go1.23.4 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN='/home/rusty/.local/go/bin'
GOCACHE='/home/rusty/.cache/go-build'
GOENV='/home/rusty/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/rusty/.local/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/rusty/.local/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/rusty/.go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/rusty/.go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.23.4'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/rusty/.config/go/telemetry'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/rusty/Documents/repo/test-http/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1074378603=/tmp/go-build -gno-record-gcc-switches'

What did you do?

If the Request is built without explicitly adding the root path it is neither identified as an error or automatically fixed.

e.g. Given:

• server.URL = http://localhost:23456
• a path with or without a root element is appended to a NewRequest using req.URL.JoinPath()
  • in both cases the resulting req.URL.Path is equal to the string given to req.URL.JoinPath() without the leading /.

What this means is that when req.Write() is called the path is added incorrectly to the HTTP request line here as something which looks like.

GET test/path HTTP/1.1

This very same line also causes the server to throw a non-descriptive 400 Bad Request.
The interesting thing here is that req.URL.String() will print correctly.

CAUSED_BY:

https://cs.opensource.google/go/go/+/master:src/net/url/url.go;l=1239-1247

Here we append the existing path to the beginnging of our elements array, and if / doesn't exist we add it. Then we strip that / back off after processing with path.Join().

Test:

package main_test

import (
	"fmt"
	"io"
	"net/http"
	"net/http/httptest"
	"testing"
)

// AppendPath appends a specified path to the request's URL.
func AppendPath(req *http.Request, s string) {
	req.URL = req.URL.JoinPath(s)
}

// Handler function for the test server
func handler(w http.ResponseWriter, r *http.Request) {
	// Respond with a simple message
	w.WriteHeader(http.StatusOK)
	w.Write([]byte("Hello, this is the test server at " + r.URL.Path))
}

// Test function to test the HTTP request
func TestHTTPRequest(t *testing.T) {
	// Create a new httptest server
	server := httptest.NewServer(http.HandlerFunc(handler))
	defer server.Close()

	// Create a new request
	req, err := http.NewRequest(http.MethodGet, server.URL, nil)
	if err != nil {
		t.Fatalf("Failed to create request: %v", err)
	}

	// Append the path to the request
	AppendPath(req, "/test/path")

	// Send the request to the test server
	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		t.Fatalf("Failed to send request: %v", err)
	}
	defer resp.Body.Close()

	// Read the response body
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		t.Fatalf("Failed to read response body: %v", err)
	}

	// Verify the response status code
	if resp.StatusCode != http.StatusOK {
		t.Fatalf("Expected status OK; got %v", resp.Status)
	}

	// Verify the response body
	expected := "Hello, this is the test server at /test/path"
	if string(body) != expected {
		t.Fatalf("Expected body %q; got %q", expected, body)
	}

	fmt.Println("Test passed!")
}

What did you see happen?

I think, I mentioned everything in the first block, but basically 400 Bad Request

What did you expect to see?

request should ensure a valid path, and then succeed or an error should be created when we attempt to create the request.

[seankhliao]

Duplicate of #58605