Yarn install quits silently if fetching one package times out

[geropl]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
yarn install quits silently if fetching one of the packages times out during fetch phase due to package size and/or network lag.
Result:

• yarn does quit before reaching linking phase, therefore no packages are installed
• No error message
• Return code 0

(Shortened) example output:

$ yarn install --verbose
yarn install v1.5.1
[...]
[1/4] Resolving packages...
warning: [...]
[2/4] Fetching packages...
verbose 1.45 Performing "GET" request to "https://registry.yarnpkg.com/envfile/-/envfile-2.3.0.tgz".
[...]
verbose 20.696 Performing "GET" request to "https://registry.yarnpkg.com/command-exists/-/command-exists-1.2.2.tgz".
$ 

If the current behavior is a bug, please provide the steps to reproduce.

1. git clone https://github.com/theia-ide/theia-apps
2. cd theia-apps/theia-java-docker
3. mv next.package.json package.json
4. yarn install

(Retry until it fails, maybe use slow internet connection)
The large package here is @theia/java, which clocks in at around 33MB.

What is the expected behavior?
yarn install prints an error message with the timeout, the package name and url it failed to fetch. Finally, it quits with a return code != 0.

Please mention your node.js, yarn and operating system version.

• node 8.10
• yarn 1.5.1
• OS: docker image node:8.10
• (Host: Ubuntu 16.04)

Update:
Maybe related: #5055

Update 2:
Neither increasing --network-timeout nor setting --mutex network helped. Interestingly, the yarn install quits long before the network timeout is due (e.g., timeout 60000ms, yarn install quits after 35s).

Update 3:
I noticed that in the cases where it fails locally (via docker run ...), my system monitor shows a very slow download (~150Kb/s on a 50 Mb/s connection). The other runs go with ~5 Mb/s).

[rally25rs]

I attempted to reproduce this by setting the timeout passed to the request lib to 1 but it does throw an error in that case:

~/Projects/yarn-test 🐒   yarndev add @theia/java
yarn add v1.6.0-0
[1/4] 🔍  Resolving packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/@theia%2fjava: ETIMEDOUT".
info If you think this is a bug, please open a bug report with the information provided in "/Users/jvalore/Projects/yarn-test/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

Not sure how else to try to repro this 🤔 maybe I can simulate a slow network somehow...

[rally25rs]

Next attempt, used Apple's "Network Link Conditioner" to simulate DSL speed. This took >2min to download that package, but still finished

~/Projects/yarn-test 🐒   yarndev add @theia/java
yarn add v1.6.0-0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
warning vscode-base-languageclient@0.0.1-alpha.3: The engine "vscode" appears to be invalid.
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved 1 new dependency.
info Direct dependencies
└─ @theia/java@0.3.7
info All dependencies
└─ @theia/java@0.3.7
✨  Done in 147.88s.

then I tried turning off my network half way through the download. That also caught the error:

~/Projects/yarn-test 🐒   yarndev add @theia/java
yarn add v1.6.0-0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
info There appears to be trouble with your network connection. Retrying...
error An unexpected error occurred: "https://registry.yarnpkg.com/@theia/java/-/java-0.3.7.tgz: connect ENETUNREACH 104.16.59.173:443 - Local (0.0.0.0:49668)".
info If you think this is a bug, please open a bug report with the information provided in "/Users/jvalore/Projects/yarn-test/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

There must be some really weird edge case here... (or maybe it only happens on linux?)

[geropl]

That's really strange. Maybe it's related to running in docker? The build is executed with jenkins in a Docker build agent, but I can run the build in docker on my laptop and reproduce it in ~20% of the runs.

Update: Dockerfile to reproduce

FROM node:8.10
COPY . .                          # <- the files from the example above
RUN yarn install
RUN ls -al node_modules

[cjbturin]

This may be a scenario to reproduce similar problem on Windows env:
#5512

[geropl]

Update 3:
I noticed that in the cases where it fails locally (via docker run ...), my system monitor shows a very slow download (~150Kb/s on a 50 Mb/s connection). The other runs go with ~5 Mb/s).

We don't use a yarn cache in our environment. Could it be that npmjs throttles/terminates repeated downloads from the same IP?

[ispringer]

Several of my teammates experienced this issue on OS X 10.13.3 with Node 5.12.0 and Yarn 1.5.1. Some comments in #764, an issue with some similarity to this one, said upgrading to Node >= 6.9.5 fixes that issue. So my teammates tried upgrading Node to the latest (9.8.0), and that did the trick! My suspicion is that the DNS caching that was recently adding to Yarn (#4447) may not work correctly with Node < 6.9.5, but it's only a hunch.

[katanacrimson]

Maybe related to #5473?

[jneves]

@rally25rs I'm still getting it regularly, is there any specific options you'd like me to try, information to provide?