Percent encode NULLs in fragments

[rmisev]

Currently URL parser removes NULL characters from fragments. But is this really needed?
My tests show that NULLs are percent encoded in most browsers (I tested URLs "https://example.com/#abc\u0000xyz" and "non-spec://example.com/#abc\u0000xyz"):

Browser	Special URL's hash	Non-special URL's hash
Chrome	#abcxyz	#abc%00xyz
Edge, IE	#abc	#abc
Firefox	#abc%00xyz	#abc%00xyz
Safari	#abc%00xyz	#abc%00xyz

So NULLs are removed in Chrome in special URLs only. In the Edge and IE a NULL character
denotes the end of string.

One reason, why Chrome removes NULLs, I found in the source code:
https://chromium.googlesource.com/chromium/src/+/refs/tags/76.0.3803.1/url/url_canon_etc.cc#304

  for (int i = ref.begin; i < end; i++) {
    if (spec[i] == 0) {
      // IE just strips NULLs, so we do too.
      continue;
    }

But as we see NULLs in Chrome and IE (and Edge) are handled differently.

I suggest to follow Firefox, Safari and percent encode NULLs in fragments.

[annevk]

Okay, so this is specifically about removing the U+0000 branch from https://url.spec.whatwg.org/#fragment-state.

This was last changed five years ago in https://www.w3.org/Bugs/Public/show_bug.cgi?id=27252 but no tests were added specifically for U+0000.

@mikewest @sleevi thoughts for Chrome?

[sleevi]

That seems reasonable to me, especially since Safari and Firefox are shipping. Perhaps @ericlaw1979 knows who to redirect this to for IE/Edge for any compat concerns?

[mikewest]

I agree with @sleevi.

[ericlaw1979]

IE/Spartan are extremely unlikely to change this.

Vis-a-vis Chrome, I'm not sure what "special URLs" include, but would this change the behavior of URLs passed out to application protocol handlers? AppProtocols tend to be the easiest routes out of the browser sandbox, and the native-code full-trust applications parsing the URLs handed to them do so with varying levels of paranoia.

[TimothyGu]

@ericlaw1979 So-called special schemes are just ftp, file, gopher, http, https, ws, wss. So it's probably not too likely for any third-party application to be affected by this. (I'm not sure about gopher though… who still uses that, and if so how?)

[mikewest]

AppProtocols tend to be the easiest routes out of the browser sandbox, and the native-code full-trust applications parsing the URLs handed to them do so with varying levels of paranoia.

I'm pretty sure that Chrome, at least, hands everything in the URL to the app (the "non-special" case in the table above). If apps are vulnerable to some aspect of parsing nulls, they're already in trouble. This wouldn't add much trouble, I think.

(I'm not sure about gopher though… who still uses that, and if so how?)

#342 :)