net/http: FileServer on WSL fails with wget and curl for files over ~4MB

[upsampled]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10.3 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build493773858=/tmp/go-build -gno-record-gcc-switches"

using WSL

Ran the same test on a CentOS Machine and I did not see this problem

What did you do?

Attempted to server files via HTTP that an application using curl can download.

package main

import (
        "log"
        "net/http"
)

func main() {
        fs := http.FileServer(http.Dir("./"))
        http.Handle("/repo/", http.StripPrefix("/repo/", fs))

        log.Println("Listening...")
        http.ListenAndServe(":80", nil)
}

What did you expect to see?

Curl to download the files.

What did you see instead?

The file server worked for file that were roughly less than 5M, but failed for larger files with curl error 18. The curl command line tool also reproduces this error:

curl http://<IP>/repo/<FILE> --output <FILE>
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                                    Dload  Upload   Total   Spent     Left  Speed
 56 7257k   56 4096k    0     0      4970k      0       0:00:01 --:--:--  0:00:01 4970k
curl: (18) transfer closed with 3237034 bytes remaining to read

Wget (busybox version) also has the same problem:

wget http://<IP>/repo/<FILE> -O <FILE>
Connecting to <IP> (<IP>:80)
pa-config-1-r0.corte   6% |**********                                                      |   460k  0:00:14 ETA
<STOPS and the file is only ~4M>

However using pythons web server, wget and curl both work
sudo python3 -m http.server 80

[bradfitz]

The WSL team want you to file bugs to them about any differences from actual Linux.

We generally don't investigate these things, or only do some when it appears we might be violating Linux or POSIX APIs in undocumented ways, or do so out of curiosity.

But please file a WSL bug and link it here.

We probably won't be investigating this. It's WSL's job to act like Linux and they don't want projects treating them like its own port.

[hullarb]

i run into the same issue and while debugging it i saw that CopyN in the fileserver returns EOF early:
https://golang.org/src/net/http/fs.go#L296
(changing that line to readall and write works just fine if you need a quick workaround)

@bradfitz i wondered if the ignored error could be checked and logged to make easier to debug issues happening there?

[gopherbot]

Change https://golang.org/cl/191971 mentions this issue: net/http: log error if fileserver fails to copy the file content

[janpfeifer]

Curiously I'm seeing the same effect. But instead of trimming the file, net/http (or something in between) changes the file somehow, starting at position 0x00400000 (exactly 4Mb) serves something different, it's very odd.

As HTTP client, I tried downloading the file in Chrome running in windows (while the server was local in WSL). And then I tried wget runnint from within WSL. Both Chrome and wget in WSL download the same altered file (the size remains the same).

go 1.15.5 here. But I've noticed that werving in WSL didn't work for a long time now.

If I serve the same file with:

python3 -m http.server -b localhost 8888

it works.

[janpfeifer]

For others bumping into this, if I change the following in my server code:

	http.Handle("/", http.FileServer(http.Dir(*flagStaticFilesPath)))

To:

	fs := dotFileHidingFileSystem{http.Dir(*flagStaticFilesPath)}
	http.Handle("/", http.FileServer(fs))

Using the dotFileHidingFileSystem implemented in the documentation's example, it works as intended.