Yarn under Git Bash replaces forward slash in package name with escaped backslash

[mcavanagh]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
When trying to add a package with a forward slash in the name using Git Bash (MinGW terminal), the forward slash is replaced by 2 backslashes

If the current behavior is a bug, please provide the steps to reproduce.

• Run yarn under the Git Bash terminal emulator on Windows 10.
• Try to add a package with a forward slash in the name. Eg: yarn add @types/rx
• Yarn tries the package name "@types\\rx" instead of "@types/rx". This happens with yarn add and yarn remove

What is the expected behavior?
The package should be successfully installed and added to the manifest.
It works successfully either by using 'npm install' in git bash, or by using the default Windows 'cmd' shell with yarn.
It feels like there's some OS-specific path separator escaping going on, rather than treating the package name purely as a string. I've tried enclosing it in quotes, but this made no difference.

Please mention your node.js, yarn and operating system version.
Node v7.6.0
Yarn v0.20.3
OS: Windows 10 v1607 (Build 14393.693)

[Daniel15]

Since you said it works in cmd.exe but fails in Git Bash, I wonder if it's Git Bash that's doing something odd.

[ksherlock]

msys2, git bash, etc plays tricks to convert path-like command line arguments. You might be able to disable with the MSYS_NO_PATHCONV environment variable -

MSYS_NO_PATHCONV=1 yarn add @types/rx

[mcavanagh]

Since you said it works in cmd.exe but fails in Git Bash, I wonder if it's Git Bash that's doing something odd.

I thought that at first, but the equivalent command using npm instead of yarn works fine

MSYS_NO_PATHCONV=1 yarn add @types/rx

No difference, unfortunately. I'm sure it's a combination issue of git bash and yarn, but as evidenced by npm working fine with the same command, I don't think it's purely a flaw with MSYS-based terminals.

[mcavanagh]

Did some digging, and I think this might be the (rather odd) solution:

In the npm shell script, the node executable is referred to as NODE_EXE="$basedir/node.exe"

In yarn, it's just node (no .exe extension)

Apparently, this actually makes a difference when running node under Git Bash (as per this stack overflow question)

I realise it's not technically a fault with yarn, but if the yarn shell script could be modified to say node.exe instead of node when running on Windows, I don't think there would be any harm done to existing cmd.exe users, and it would line up with how npm works, making it work under Git Bash too.

[Daniel15]

Hmm that's tricky. The same shell script is used across all platforms (Linux, Mac OS, Windows), which is why it just calls node rather than node.exe. This is the script: https://github.com/yarnpkg/yarn/blob/master/bin/yarn

if the yarn shell script could be modified to say node.exe instead of node when running on Windows

Two possible approaches I can think of:

1. Detect Windows in the script itself. It looks like there's already some custom logic for MSYS and MINGW terminals, maybe we just need a bit more custom logic in there.
2. Modify the script that builds the Windows installer (https://github.com/yarnpkg/yarn/blob/master/resources/winsetup/YarnSetup.wixproj#L43) to edit the shell script and replace node with node.exe.

Feel free to send a pull request 👍

[mcavanagh]

The npm shell script does the following:

NODE_EXE="$basedir/node.exe"
if ! [ -x "$NODE_EXE" ]; then
  NODE_EXE=node
fi

Would this be an acceptable practice in yarn? If so, we wouldn't need to alter the windows installer config (although it could be argued that it should be done there, as it's windows-specific)

Also, I've just realised the reason why it all works fine under cmd.exe is that when you type yarn there, it's running yarn.cmd, not the shell script (obviously)

Feel free to send a pull request

I forked the repo so that I could open a pull request, but I wanted to be sure that none of the tests are affected, so I ran yarn run test first (before altering anything, and under windows cmd.exe) expecting 100% passes and it came up with quite a few failures in a few categories. This was from the master branch of the current yarn repo. Is this normal, and if so, how can I know if my changes broke something?

The final test summary was:

Test Suites: 5 failed, 46 passed, 51 total
Tests:       10 failed, 6 skipped, 411 passed, 427 total
Snapshots:   50 passed, 50 total
Time:        251.26s, estimated 261s
Ran all test suites.

Example errors:

 FAIL  __tests__\commands\add.js (247.131s)
  ● add should put a git dependency to mirror

    Error: Refusing to download the git repo "https://github.com/jshttp/mime-db.git" over HTTPS without a commit hash - possible certificate error?

 FAIL  __tests__\package-resolver.js (67.031s)
  ● resolve https://git@github.com/stevemao/left-pad.git

    do nothing

      at src\util\git.js:137:17
          at Generator.next (<anonymous>)
      at step (node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
      at node_modules\babel-runtime\helpers\asyncToGenerator.js:28:13
      at process._tickCallback (internal\process\next_tick.js:103:7)

  ● resolve https://bitbucket.org/hgarcia/node-bitbucket-api.git

    do nothing

      at src\util\git.js:137:17
          at Generator.next (<anonymous>)
      at step (node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
      at node_modules\babel-runtime\helpers\asyncToGenerator.js:28:13
      at process._tickCallback (internal\process\next_tick.js:103:7)

  ● resolve https://github.com/yarnpkg/yarn/releases/download/v0.18.1/yarn-v0.18.1.tar.gz

    Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.

      at Timeout._onTimeout (node_modules\jest-jasmine2\vendor\jasmine-2.5.2.js:1977:23)
      at ontimeout (timers.js:365:14)
      at tryOnTimeout (timers.js:237:5)
      at Timer.listOnTimeout (timers.js:207:5)

  ● resolve https://github.com/yarnpkg/e2e-test-repo.git#greenkeeper/cross-env-3.1.4

    do nothing

      at src\util\git.js:137:17
          at Generator.next (<anonymous>)
      at step (node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
      at node_modules\babel-runtime\helpers\asyncToGenerator.js:28:13
      at process._tickCallback (internal\process\next_tick.js:103:7)

  ● resolve gitlab:leanlabsio/kanban

    Couldn't find the binary git

      at ChildProcess.proc.on.err (src\util\child.js:42:18)
      at emitOne (events.js:96:13)
      at ChildProcess.emit (events.js:189:7)
      at Process.ChildProcess._handle.onexit (internal\child_process.js:213:12)
      at onErrorNT (internal\child_process.js:359:16)
      at _combinedTickCallback (internal\process\next_tick.js:74:11)
      at process._tickCallback (internal\process\next_tick.js:98:9)

  ● resolve gist:d59975ac23e26ad4e25b

    do nothing

      at src\util\git.js:137:17
          at Generator.next (<anonymous>)
      at step (node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
      at node_modules\babel-runtime\helpers\asyncToGenerator.js:28:13
      at process._tickCallback (internal\process\next_tick.js:103:7)

  × resolve https://git@github.com/stevemao/left-pad.git (151ms)
  × resolve https://bitbucket.org/hgarcia/node-bitbucket-api.git (72ms)
  × resolve https://github.com/yarnpkg/yarn/releases/download/v0.18.1/yarn-v0.18.1.tar.gz (60001ms)
  × resolve https://github.com/yarnpkg/e2e-test-repo.git#greenkeeper/cross-env-3.1.4 (18ms)
  × resolve gitlab:leanlabsio/kanban (73ms)
  × resolve gist:d59975ac23e26ad4e25b (139ms)

[Daniel15]

Would this be an acceptable practice in yarn?

Yeah, that should be fine!

I ran yarn run test first (before altering anything, and under windows cmd.exe) expecting 100% passes and it came up with quite a few failures in a few categories. This was from the master branch of the current yarn repo.

Hmm, that's strange... We do run the test suite on Windows, see https://ci.appveyor.com/project/kittens/yarn/history so they should be passing. Some of them look a bit flaky now, though.

[mcavanagh]

Hmm, that's strange... We do run the test suite on Windows

Never mind, just realised it's my windows git setup that's the problem (because I always use Git Bash for git, hence this entire situation :)

Tests pass under git bash. I'll make a pull request.

[mcavanagh]

Ok, looks like there's a new wrinkle to this:

If yarn's shell script detects you're running under MINGW or MSYS, it runs winpty node "$basedir/yarn.js" "$@"

It seems like winpty also has an effect on how node deals with forward slashes in its args. To summarize:

> winpty node bin/yarn.js add @types/rx fails
> winpty node.exe bin/yarn.js add @types/rx fails
> node bin/yarn.js add @types/rx fails
> node.exe bin/yarn.js add @types/rx WORKS

So specifying '.exe' on the end only makes a difference when NOT using winpty. And as it stands, winpty will always get used if you're using MINGW / MSYS on windows.
Not sure what the correct way around this is. Any thoughts?

[Daniel15]

The winpty stuff was added in #743 to fix another issue. Maybe the person that added it (@thetrompf) has some ideas.

[thetrompf]

Hmm.. will look into this today or tomorrow

I'm encountering the same issue using Yarn v0.21.3.

Platform:

• Yarn v0.21.3
• Node v6.10.0
• Windows 7

I get the following error:

$ yarn add @types/debug--dev
yarn add v0.21.3
[1/4] Resolving packages...
error Couldn't find package "@types\\debug" on the "npm" registry.

I'm running the add command within cmder using Git for Windows Bash shell.

uname -s returns MINGW64_NT-6.1

The execution path for me goes through the call to winpty node "$basedir/yarn.js" "$@".

Yarn works correctly within Windows cmd shell, though that's because it doesn't go through winpty.

I've temporarily removed winpty from my installed version of Yarn, and the installation works correctly.

To debug the issue I began downgrading my Yarn installation. The oldest version of Yarn that works is v0.18.2. Beginning with v0.19.1 (I did not try v0.19.0), the issue is encountered. v0.19.0 introduced 8515f9b

[GemsbokVis]

I'm encountering the same issue using Yarn v0.21.3

Platform:

Yarn v0.21.3
Node v6.10.0
Windows 10

The one thing I did manage is that it works under cmd.exe so I'm presuming that it has to do with with the path using backslash instead of forward slash as with Git Bash.