net/url: ParseRequestURI changes behavior from 1.9 with IRC URLs

[bradfitz]

Running Go 1.10 inside of Google revealed that this test now fails in Go 1.10:

https://github.com/asaskevich/govalidator

validator_test.go:698: Expected IsRequestURL("irc://#channel@network") to be true, got false

... due to ba1018b (https://go-review.googlesource.com/87038)

For #23392

Is that a valid URL?

[adamdecaf]

Is it assumed that #channel is the authority (username)? Section 3.2 Authority mentions this:

The authority component is preceded by a double slash ("//") and is
terminated by the next slash ("/"), question mark ("?"), or number
sign ("#") character, or by the end of the URI.

Edit: Looking at a couple other languages (ruby, java) these seem to parse that input as a fragment.

scala> import java.net.URI

scala> new URI("irc://#channel@network")
res1: java.net.URI = irc://#channel@network

scala> res1.getFragment
res2: String = channel@network

adam@~$ irb
irb(main):003:0> require 'uri'

irb(main):008:0> URI.parse("irc://#channel@network").fragment   
=> "channel@network"

[rsc]

Note that this is about ParseRequestURI (IsRequestURL is in the govalidator package and calls url.ParseRequestURI).
The comparison with java.net.URI and ruby uri are off-center - they are more like url.Parse, not url.ParseRequestURI. url.Parse does accept irc://#channel@network and take the #channel@network as a fragment. But url.ParseRequestURI's whole reason for existing is to not look for fragments.

The relevant docs say:

// ParseRequestURI parses rawurl into a URL structure. It assumes that
// rawurl was received in an HTTP request, so the rawurl is interpreted
// only as an absolute URI or an absolute path.
// The string rawurl is assumed not to have a #fragment suffix.
// (Web browsers strip #fragment before sending the URL to a web server.)

And of course #channel@network is in fact NOT a valid user info. In this program:

package main

import (
	"fmt"
	"net/url"
)

func main() {
	u, err := url.Parse("irc://#channel@network")
	fmt.Printf("%+v %q %v\n", u, u.Fragment, err)
	u, err = url.ParseRequestURI("irc://#channel@network")
	fmt.Printf("%+v %v\n", u, err)
}

url.Parse was happy in Go 1.9 and Go 1.10. url.ParseRequestURI was happy in Go 1.9 but silently converted the URL, when reprinting it with u.String, into irc://%23channel@network. If everyone wants the channel@network to be a fragment, that %23 is going to hurt.

It seems like this change is unfortunate and will break some code, but Go 1.10 is actually doing the right thing by rejecting # in userinfo, and Go 1.9 and earlier were not.

[rsc]

Even govalidator says:

// IsRequestURL check if the string rawurl, assuming
// it was received in an HTTP request, is a valid
// URL confirm to RFC 3986

I don't believe that receiving irc://#channel@network in an HTTP request is valid.

[bradfitz]

@asaskevich, can you fix govalidator?

[rsc]

Took me a few minutes but I think @bradfitz means fix the govalidator tests not to expect IsRequestURL("irc://#channel@network") to be true.

[magical]

Is that even a valid IRC url? The channel usually goes in the path, like irc://network/channel. I've never seen irc://#channel@network.

[rsc]

irc://#channel@network is a fine URL, it's just not a fine request URL, that is, something you'd see in an HTTP request line. I don't see anyone arguing otherwise here, so I'm going to close this as working as intended.