Can't access remote repository if the URL contains Chinese word

[YueLinHo]

• [v] I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

64 bit

$ git --version --build-options
git version 2.10.2.windows.1

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver
Microsoft Windows [版本 6.1.7601]

Win7 64 Pro

• What options did you set as part of the installation? Or did you choose the defaults?

> type "C:\Program Files\Git\etc\install-options.txt"
Path Option: Cmd
SSH Option: OpenSSH
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled

• Any other interesting things about your environment that might be related to the issue you're seeing?

Suppose 2.10.1 has the same issue.

Study some, and compare the changes between 2.10.0 and 2.10.1, then I guess the changes of url.c in
commit introduce hex2chr() for converting two hexadecimal digits to a character may be the key?

Also see TortoiseGit issue 2859

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

CMD

• What commands did you run to trigger this issue? If you can provide a Minimal, Complete, and Verifiable example this will help us understand the issue.

Can't clone if URL contains Chinese word(s)

$ git.exe clone --progress -v "D:\Temp\i2859\Test中文.git" "D:\Temp\i2859\Test中文2.10.2"

Or add a remote and its URL contains Chinese word(s). Then can't perform fetch/push anymore.

Note: English URL works fine.

• What did you expect to occur after running these commands?

Like Git Bash does:

$ git.exe clone --progress -v "D:\Temp\i2859\Test中文.git" "D:\Temp\i2859\Test中文2.10.2"
Cloning into 'D:\Temp\i2859\Test中文2.10.2'...
done.

• What actually happened instead?

D:\Temp\i2589>git.exe clone --progress -v "D:\Temp\i2589\Test中文.git" "D:\Temp\i2589\Test中文2.10.2"
Cloning into 'D:\Temp\i2589\Test中文2.10.2'...
"git-upload-pack 'D:\Temp\i2589\Test中�.git'": git-upload-pack 'D:\Temp\i2589\Test中�.git': No such file or directory
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Warning: Your console font probably doesn't support Unicode. If you experience strange characters in the output, consider switching to a TrueType font such as Consolas!

Update 1

TortoiseGit interface shows those two strings normally:

And test with true type in cmd.exe:

D:\Temp\i2589>git clone "D:\Temp\i2859\Test中文.git" "D:\Temp\i2859\中文2.10.2"
Cloning into 'D:\Temp\i2859\中文2.10.2'...
"git-upload-pack 'D:\Temp\i2859\Test中�.git'": git-upload-pack 'D:\Temp\i2859
\Test中�.git': No such file or directory
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

That warning is gone.

Update 2

Other testing results: (TortoiseGit calls git.exe)

As you can see:

1. 2.10.0 is last good for TortoiseGit
2. 2.10.1 is first bad. Bash works fine, but CMD. (based on that TortoiseGit calls git.exe, and the result in my last comment)

Workaround

call set LC_ALL=C in cmd.exe

[PhilipOakley]

----- Original Message -----
From: Yue Lin Ho
To: git-for-windows/git
Sent: Sunday, November 06, 2016 4:08 PM
Subject: [git-for-windows/git] Can't access remote repository if the URL contains Chinese word (#945)

a.. [v] I was not able to find an open or closed issue matching what I'm seeing 

Setup
a.. Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
$ git --version --build-options
git version 2.10.2.windows.1
a.. Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
$ cmd.exe /c ver
Microsoft Windows [版本 6.1.7601]
a.. What options did you set as part of the installation? Or did you choose the
defaults?

One of the following:

type "C:\Program Files\Git\etc\install-options.txt"
type "C:\Program Files (x86)\Git\etc\install-options.txt"
type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt
Path Option: Cmd
SSH Option: OpenSSH
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled
a.. Any other interesting things about your environment that might be related
to the issue you're seeing?
Suppose 2.10.1 has the same issue.

Study some, and compare the changes between 2.10.0 and 2.10.1, then I guess the changes of url.c in
commit introduce hex2chr() for converting two hexadecimal digits to a character may be the key?

Also see TortoiseGit issue 2859

Details
a.. Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other
CMD

a.. What commands did you run to trigger this issue? If you can provide a
Minimal, Complete, and Verifiable example
this will help us understand the issue. 

Add a remote and its URL contains Chinese word(s). Then can't perform clone/fetch/push anymore.

Note: English URL works fine.

a.. What did you expect to occur after running these commands? 

Like Git Bash does:

$ git.exe clone --progress -v "D:\Temp\i2859\Test中文.git" "D:\Temp\i2859\Test中文2.10.2"
Cloning into 'D:\Temp\i2859\Test中文2.10.2'...
done.
a.. What actually happened instead?
D:\Temp\i2589>git.exe clone --progress -v "D:\Temp\i2589\Test中文.git" "D:\Temp\i2589\Test中文2.10.2"
Cloning into 'D:\Temp\i2589\Test中文2.10.2'...
"git-upload-pack 'D:\Temp\i2589\Test中�.git'": git-upload-pack 'D:\Temp\i2589\Test中�.git': No such file or directory
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Warning: Your console font probably doesn't support Unicode. If you experience strange characters in the output, consider switching to a TrueType font such as Consolas!

hello,

I think this warning message "Your console font probably doesn't support Unicode. If you experience strange characters in the output, consider switching to a TrueType font such as Consolas!" Is at the heart of the problem.

It looks like an encoding problem. Git assumes you use UTF-8 unicode. I guess that is not how your chinese characters/words are being encoded when passed to git, so while they are displayed correctly on screen, they are different 8-bit byte sequences.

Try the suggestion of "switching to a TrueType font such as Consolas" and seeing if that has an effect.

Philip

[YueLinHo]

@PhilipOakley

Hi,

I test it again with true type:

D:\Temp\i2589>git clone "D:\Temp\i2859\Test中文.git" "D:\Temp\i2859\中文2.10.2"
Cloning into 'D:\Temp\i2859\中文2.10.2'...
"git-upload-pack 'D:\Temp\i2859\Test中�.git'": git-upload-pack 'D:\Temp\i2859
\Test中�.git': No such file or directory
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

That warning is gone.

Note: Git Bash works fine, but Windows Command Prompt.

[YueLinHo]

Post earlier testing results: (TortoiseGit calls git.exe)

As you can see:

1. 2.10.0 is last good for TortoiseGit
2. 2.10.1 is first bad. Bash works fine, but CMD. (based on that TortoiseGit calls git.exe, and the result in my last comment)

So, I compared the code between tag v2.10.0.windows.1 (commit 1f448f5) and v2.10.1.windows.1 (commit 3374789), and my first guessing is the changes of url.c in commit introduce hex2chr() for converting two hexadecimal digits to a character. But, I am not sure it is the root cause.

[dscho]

Please call set LC_ALL=C in cmd.exe and try to clone again.

[YueLinHo]

@dscho OH~ YES~ set LC_ALL=C works. (Oh~ Happy~~ ^____^)

Thank you!!!!!

[PhilipOakley]

Hi,

I still suspect that it's an encoding issue.

Your results show that this is not a problem for GfW 2.10.1/2 from git bash, so we are doing something right.

The results then show that when TortoiseGit does the command then it fails. So it will be somewhere in the tortoiseGit interface. (with the same suspect that it is a 'bad' code page / encoding issue).

Can you capture the two strings as byte sequences to see if they follow the UTF-8 and code page encodings?

P.
----- Original Message -----
From: Yue Lin Ho
To: git-for-windows/git
Cc: Philip Oakley ; Mention
Sent: Monday, November 07, 2016 9:35 AM
Subject: Re: [git-for-windows/git] Can't access remote repository if the URL contains Chinese word (#945)

Post earlier testing result: (TortoiseGit calls git.exe)

As you can see:

1. 2.10.0 is last good for TortoiseGit

2. Bash works fine, but CMD (based on that TortoiseGit calls git.exe, and the result in my last comment).

   So, I compared the code between tag v2.10.0.windows.1 (commit 1f448f5) and v2.10.1.windows.1 (commit 3374789), and my first guessing is the changes of url.c in commit introduce hex2chr() for converting two hexadecimal digits to a character. But, I am not sure it is the root cause.

   —
   You are receiving this because you were mentioned.
   Reply to this email directly, view it on GitHub, or mute the thread.

[YueLinHo]

@PhilipOakley

I still suspect that it's an encoding issue.
Your results show that this is not a problem for GfW 2.10.1/2 from git bash, so we are doing something right.
The results then show that when TortoiseGit does the command then it fails. So it will be somewhere in the tortoiseGit interface. (with the same suspect that it is a 'bad' code page / encoding issue).

Can you capture the two strings as byte sequences to see if they follow the UTF-8 and code page encodings?

TortoiseGit interface shows those two strings normally:

So, I guess it follows. :-)

[YueLinHo]

I can build Git for Windows and its installer now. Then, I tested the installer which revert the change by the commit I suspected, say commit d233097. The result is that clone still does not work. So, I think that that commit can be excluded. :-)

[YueLinHo]

I know Git for Windows can be built with VS2015 since 2.10.2. So, my next step may be debug it with VS2015? right?

[YueLinHo]

I built 2.10.0 and made a installer, then can't clone. XD

Looks like it's not source code problem. Any idea?

[PhilipOakley]

I build 2.10.0 and make a installer, then can't clone. XD
Looks like it's not source code problem. Any idea?

It maybe that your build completed, and overwrote your current install (within the sdk), and that there is some issue with your build.

Does your regular Git still work.

[dscho]

@YueLinHo is there a chance to teach TortoiseGit to set LC_ALL=C? Or would it help if Git for Windows' cmd\git.exe would set LC_ALL unless it was set already?

[YueLinHo]

@dscho

is there a chance to teach TortoiseGit to set LC_ALL=C?

I am not sure. I guess TortoiseGit can do so. I will find time to try it. (@csware Can TortoiseGit do it?)
For me, I added a env. var. LC_ALL=C, so that I can use TortoiseGit in daily work.

Or would it help if Git for Windows' cmd\git.exe would set LC_ALL unless it was set already?

I think it would help. ^_^