Regression: 2.20.1 git-upload-pack SPNEGO transaction breakage

[kkm000]

• I was able to find an open issue git curl fails when remote terminates unexpectedly #2012 very similar to what I'm seeing, but it is marked "unclear".

Setup

• A Git remote on a Kerberized HTTPS server with "smart" Git protocol support (GitLab EE).
• A domain-joined Windows 10.0.17763.253 machine.
• GfW 2.20.1 64-bit. 2.20.0 seems ok.

The issue reproduces with GfW either installed or Portable.

$ git --version --build-options
git version 2.20.1.windows.1
cpu: x86_64
built from commit: 7c9fbc07db0e2939b36095df45864b8cda19b64f
sizeof-long: 4
sizeof-size_t: 8

$ git --exec-path
C:/Users/kkm/Downloads/PortableGit-2.20.1-64-bit.7z/mingw64/libexec/git-core

$ git config --get http.postbuffer
100000000

$ git config --get http.sslbackend
schannel

Details

A developer upgraded to GfW 2.20.1 (from an unknown version) and reported that he was unable to push to most, but not all, of his repositories. The error happened consistently with certain repositories: if it failed, it always failed. Upon examination, a fetch from most (but not all) repositories on the same server also failed:

$ git fetch --all                                                                                               
Fetching origin                                                                                                 
error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500 Internal Server Error                 
fatal: the remote end hung up unexpectedly                                                                      
error: Could not fetch origin                                                                                   
Fetching my
 (No response--second remote succeeded)

A downgrade to 2.18.0 fixed his problem. Back on my machine, I tried to reproduce the issue. It seems that it occurs only when there is actual data to fetch; a repo that has been successfully fetched with 2.20.0 does not expose this behavior, but when Git actually wants some commits from the server, the error occurs.

I am not yet sure if 2.20.0 not broken, but it seems that it is out in the clean. I am going to bite the bullet and upgrade to it, and see if the same thing happens. But GfW 2.20.1 is certainly unhealthy.

Trace analysis

Here is the important part of traces (let me know if this is not enough). Domain name, IP addresses and repository name have been redacted. Long Authorization: headers have been truncated for readability. Also, I trimmed non-essential nearly identical repeating log lines, clearly marked.

First, there is a successful GET request for the remote's refs.

The error happens on the POST request to the /git-upload-pack endpoint. The Content-Length: header is set to a non-zero length on the initial request (responded to with a 401), but is zero in the second, authenticating request.

$ GIT_TRACE=2 GIT_TRACE_PACKET=2 GIT_CURL_VERBOSE=1 git clone https://gitlab.domain-name.local/sa-apps/repository-name.git
20:22:10.132799 exec-cmd.c:236          trace: resolved executable dir: C:/Users/kkm/Downloads/PortableGit-2.20.1-64-bit.7z/mingw64/bin
20:22:10.133799 git.c:418               trace: built-in: git clone https://gitlab.domain-name.local/sa-apps/repository-name.git
Cloning into 'repository-name'...
20:22:10.167842 run-command.c:643       trace: run_command: git remote-https origin https://gitlab.domain-name.local/sa-apps/repository-name.git
20:22:10.175842 exec-cmd.c:236          trace: resolved executable dir: C:/Users/kkm/Downloads/PortableGit-2.20.1-64-bit.7z/mingw64/libexec/git-core
20:22:10.177842 git.c:675               trace: exec: git-remote-https origin https://gitlab.domain-name.local/sa-apps/repository-name.git
20:22:10.177842 run-command.c:643       trace: run_command: git-remote-https origin https://gitlab.domain-name.local/sa-apps/repository-name.git
20:22:10.187843 exec-cmd.c:236          trace: resolved executable dir: C:/Users/kkm/Downloads/PortableGit-2.20.1-64-bit.7z/mingw64/libexec/git-core
*   Trying 10.NN.NN.NN...
* TCP_NODELAY set
* Connected to gitlab.domain-name.local (10.NN.NN.NN) port 443 (#0)
* schannel: SSL/TLS connection with gitlab.domain-name.local port 443 (step 1/3)
* schannel: checking server certificate revocation
* schannel: sending initial handshake data: sending 178 bytes...
* schannel: sent initial handshake data: sent 178 bytes
* schannel: SSL/TLS connection with gitlab.domain-name.local port 443 (step 2/3)
* schannel: failed to receive handshake, need more data
* schannel: SSL/TLS connection with gitlab.domain-name.local port 443 (step 2/3)
* schannel: encrypted data got 1937
* schannel: encrypted data buffer: offset 1937 length 4096
* schannel: sending next handshake data: sending 126 bytes...
* schannel: SSL/TLS connection with gitlab.domain-name.local port 443 (step 2/3)
* schannel: encrypted data got 258
* schannel: encrypted data buffer: offset 258 length 4096
* schannel: SSL/TLS handshake complete
* schannel: SSL/TLS connection with gitlab.domain-name.local port 443 (step 3/3)
* schannel: stored credential handle in session cache
> GET /sa-apps/repository-name.git/info/refs?service=git-upload-pack HTTP/1.1
Host: gitlab.domain-name.local
User-Agent: git/2.20.1.windows.1
Accept: */*
Accept-Encoding: deflate, gzip
Pragma: no-cache

* schannel: client wants to read 16384 bytes
* schannel: encdata_buffer resized 17408
* schannel: encrypted data buffer: offset 0 length 17408
[... more similar encrypt/decrypt lines ...]
* schannel: decrypted data buffer: offset 474 length 16384
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 474
* schannel: decrypted data buffer: offset 0 length 16384
< HTTP/1.1 401 Unauthorized
< Server: nginx
< Date: Wed, 16 Jan 2019 04:22:09 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 26
< Connection: keep-alive
< Cache-Control: no-cache
< WWW-Authenticate: Basic realm="GitLab"
< WWW-Authenticate: Negotiate
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-Request-Id: 5cb6c5c1-5756-4366-9d0f-a41a22e5fdde
< X-Runtime: 0.029198
< X-Ua-Compatible: IE=edge
< X-Xss-Protection: 1; mode=block
<
* Ignoring the response-body
* Connection #0 to host gitlab.domain-name.local left intact
* Issue another request to this URL: 'https://gitlab.domain-name.local/sa-apps/repository-name.git/info/refs?service=git-upload-pack'
* Found bundle for host gitlab.domain-name.local: 0x15128c0 [can pipeline]
* Could pipeline, but not asked to!
* Re-using existing connection! (#0) with host gitlab.domain-name.local
* Connected to gitlab.domain-name.local (10.NN.NN.NN) port 443 (#0)
> GET /sa-apps/repository-name.git/info/refs?service=git-upload-pack HTTP/1.1
Host: gitlab.domain-name.local
User-Agent: git/2.20.1.windows.1
Accept: */*
Accept-Encoding: deflate, gzip
Pragma: no-cache

* schannel: client wants to read 16384 bytes
* schannel: encrypted data buffer: offset 0 length 17408
[... more similar encrypt/decrypt lines ...]
* schannel: decrypted data buffer: offset 474 length 16384
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 474
* schannel: decrypted data buffer: offset 0 length 16384
< HTTP/1.1 401 Unauthorized
< Server: nginx
< Date: Wed, 16 Jan 2019 04:22:10 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 26
< Connection: keep-alive
< Cache-Control: no-cache
< WWW-Authenticate: Basic realm="GitLab"
< WWW-Authenticate: Negotiate
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-Request-Id: 34424fe5-ea0f-4c53-9624-bc68804e6b09
< X-Runtime: 0.024844
< X-Ua-Compatible: IE=edge
< X-Xss-Protection: 1; mode=block
<
* Ignoring the response-body
* Connection #0 to host gitlab.domain-name.local left intact
* Issue another request to this URL: 'https://gitlab.domain-name.local/sa-apps/repository-name.git/info/refs?service=git-upload-pack'
* Found bundle for host gitlab.domain-name.local: 0x15128c0 [can pipeline]
* Could pipeline, but not asked to!
* Re-using existing connection! (#0) with host gitlab.domain-name.local
* Connected to gitlab.domain-name.local (10.NN.NN.NN) port 443 (#0)
* Server auth using Negotiate with user ''
> GET /sa-apps/repository-name.git/info/refs?service=git-upload-pack HTTP/1.1
Host: gitlab.domain-name.local
Authorization: Negotiate YIIG/gY[...snip...]
User-Agent: git/2.20.1.windows.1
Accept: */*
Accept-Encoding: deflate, gzip
Pragma: no-cache

* schannel: client wants to read 16384 bytes
* schannel: encrypted data buffer: offset 0 length 17408
[... more similar encrypt/decrypt lines ...]
* schannel: decrypted data buffer: offset 4133 length 16384
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 4133
* schannel: decrypted data buffer: offset 0 length 16384
< HTTP/1.1 200 OK
< Server: nginx
< Date: Wed, 16 Jan 2019 04:22:10 GMT
< Content-Type: application/x-git-upload-pack-advertisement
< Transfer-Encoding: chunked
< Connection: keep-alive
< Cache-Control: no-cache
< Www-Authenticate: Negotiate oYG2MIG[...snip...]
< Strict-Transport-Security: max-age=31536000
<
* schannel: client wants to read 16384 bytes
* schannel: encrypted data buffer: offset 0 length 17408
* schannel: encrypted data got 2389
[... more similar encrypt/decrypt lines ...]
* schannel: decrypted data buffer: offset 2331 length 16384
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 2331
* schannel: decrypted data buffer: offset 0 length 16384
* Closing connection 0
* schannel: shutting down SSL/TLS connection with gitlab.domain-name.local port 443
* schannel: clear security context handle
20:22:10.340841 pkt-line.c:80           packet:          git< # service=git-upload-pack
20:22:10.340841 pkt-line.c:80           packet:          git< 0000
20:22:10.340841 pkt-line.c:80           packet:          git< 18b33246a5f3f2676c1dcf6a4935565cb5ceec5c HEAD\0multi_ack thin-pack side-band side-band-64k ofs-delta shallow deepen-since deepen-not deepen-relative no-progress include-tag multi_ack_detailed no-done symref=HEAD:refs/heads/dev agent=git/2.18.1
20:22:10.340841 pkt-line.c:80           packet:          git< 18b33246a5f3f2676c1dcf6a4935565cb5ceec5c refs/heads/dev
20:22:10.340841 pkt-line.c:80           packet:          git< b9c9cd8343065a47a42ebd5185ff82ba655f358a refs/heads/master
 [... more refs ...]
20:22:10.341841 pkt-line.c:80           packet:          git< b9c9cd8343065a47a42ebd5185ff82ba655f358a refs/tags/v106^{}
20:22:10.341841 pkt-line.c:80           packet:          git< 0000
20:22:10.345841 pkt-line.c:80           packet:          git> 18b33246a5f3f2676c1dcf6a4935565cb5ceec5c refs/heads/dev
 [... more refs ...]
20:22:10.345841 pkt-line.c:80           packet:          git> fb9f037951b47f6a8d5faf7afbfc98309d75608b refs/tags/v106
20:22:10.345841 pkt-line.c:80           packet:          git> 0000
20:22:10.345841 run-command.c:643       trace: run_command: git fetch-pack --stateless-rpc --stdin --lock-pack --thin --check-self-contained-and-connected --cloning https://gitlab.domain-name.local/sa-apps/repository-name.git/
20:22:10.354841 exec-cmd.c:236          trace: resolved executable dir: C:/Users/kkm/Downloads/PortableGit-2.20.1-64-bit.7z/mingw64/libexec/git-core
20:22:10.357840 git.c:418               trace: built-in: git fetch-pack --stateless-rpc --stdin --lock-pack --thin --check-self-contained-and-connected --cloning https://gitlab.domain-name.local/sa-apps/repository-name.git/
20:22:10.357840 pkt-line.c:80           packet:   fetch-pack< 18b33246a5f3f2676c1dcf6a4935565cb5ceec5c refs/heads/dev
 [... more refs ...]
20:22:10.357840 pkt-line.c:80           packet:   fetch-pack< fb9f037951b47f6a8d5faf7afbfc98309d75608b refs/tags/v106
20:22:10.357840 pkt-line.c:80           packet:   fetch-pack< 0000
20:22:10.357840 pkt-line.c:80           packet:   fetch-pack< 18b33246a5f3f2676c1dcf6a4935565cb5ceec5c HEAD\0multi_ack thin-pack side-band side-band-64k ofs-delta shallow deepen-since deepen-not deepen-relative no-progress include-tag multi_ack_detailed no-done symref=HEAD:refs/heads/dev agent=git/2.18.1
20:22:10.357840 pkt-line.c:80           packet:   fetch-pack< 18b33246a5f3f2676c1dcf6a4935565cb5ceec5c refs/heads/dev
 [... more refs ...]
20:22:10.358842 pkt-line.c:80           packet:   fetch-pack< b9c9cd8343065a47a42ebd5185ff82ba655f358a refs/tags/v106^{}
20:22:10.358842 pkt-line.c:80           packet:   fetch-pack< 0000
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> want 18b33246a5f3f2676c1dcf6a4935565cb5ceec5c multi_ack_detailed no-done side-band-64k thin-pack ofs-delta deepen-since deepen-not agent=git/2.20.1.windows.1
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> want b9c9cd8343065a47a42ebd5185ff82ba655f358a
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> want 190a26330eb8411c577e987eeb9d8698330435f6
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> want c50efc4f837d9c3a460b2e883554806c76926345
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> want 62863160bbdff72206efa4454663e447580b07b0
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> want 7870c7424997a8c90d14c6b36b41dad27f9819c9
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> want 6f10ae66bb05379ffcde14b268aa1cf2482939be
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> want 8a24e2673a258e9f47c481946901702ba20eee06
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> want 207c35413635f7930a8d900988051065eb98b202
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> want fb9f037951b47f6a8d5faf7afbfc98309d75608b
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> 0000
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> done
20:22:10.363840 pkt-line.c:80           packet:   fetch-pack> 0000
20:22:10.363840 pkt-line.c:80           packet:          git< 00a2want 18b33246a5f3f2676c1dcf6a4935565cb5ceec5c multi_ack_detailed no-done side-band-64k thin-pack ofs-delta deepen-since deepen-not agent=git/2.20.1.windows.10032want b9c9cd8343065a47a42ebd5185ff82ba655f358a0032want 190a26330eb8411c577e987eeb9d8698330435f60032want c50efc4f837d9c3a460b2e883554806c769263450032want 62863160bbdff72206efa4454663e447580b07b00032want 7870c7424997a8c90d14c6b36b41dad27f9819c90032want 6f10ae66bb05379ffcde14b268aa1cf2482939be0032want 8a24e2673a258e9f47c481946901702ba20eee060032want 207c35413635f7930a8d900988051065eb98b2020032want fb9f037951b47f6a8d5faf7afbfc98309d75608b00000009done
20:22:10.363840 pkt-line.c:80           packet:          git< 0000
* NTLM-proxy picked AND auth done set, clear picked!
* Hostname gitlab.domain-name.local was found in DNS cache
*   Trying 10.NN.NN.NN...
* TCP_NODELAY set
* Connected to gitlab.domain-name.local (10.NN.NN.NN) port 443 (#1)
* schannel: SSL/TLS connection with gitlab.domain-name.local port 443 (step 1/3)
* schannel: re-using existing credential handle
* schannel: incremented credential handle refcount = 2
* schannel: sending initial handshake data: sending 370 bytes...
* schannel: sent initial handshake data: sent 370 bytes
* schannel: SSL/TLS connection with gitlab.domain-name.local port 443 (step 2/3)
* schannel: failed to receive handshake, need more data
* schannel: SSL/TLS connection with gitlab.domain-name.local port 443 (step 2/3)
* schannel: encrypted data got 105
* schannel: encrypted data buffer: offset 105 length 4096
* schannel: sending next handshake data: sending 51 bytes...
* schannel: SSL/TLS handshake complete
* schannel: SSL/TLS connection with gitlab.domain-name.local port 443 (step 3/3)
> POST /sa-apps/repository-name.git/git-upload-pack HTTP/1.1
Host: gitlab.domain-name.local
User-Agent: git/2.20.1.windows.1
Accept-Encoding: deflate, gzip
Content-Type: application/x-git-upload-pack-request
Accept: application/x-git-upload-pack-result
Content-Length: 625

* upload completely sent off: 625 out of 625 bytes

Let's pause here for a sec. Content-Length: is set to 625.

* schannel: client wants to read 16384 bytes
* schannel: encdata_buffer resized 17408
* schannel: encrypted data buffer: offset 0 length 17408
* schannel: encrypted data got 503
* schannel: encrypted data buffer: offset 503 length 17408
* schannel: decrypted data length: 474
* schannel: decrypted data added: 474
* schannel: decrypted data cached: offset 474 length 16384
* schannel: encrypted data buffer: offset 0 length 17408
* schannel: decrypted data buffer: offset 474 length 16384
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 474
* schannel: decrypted data buffer: offset 0 length 16384
< HTTP/1.1 401 Unauthorized
< Server: nginx
< Date: Wed, 16 Jan 2019 04:22:10 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 26
< Connection: keep-alive
< Cache-Control: no-cache
< WWW-Authenticate: Basic realm="GitLab"
< WWW-Authenticate: Negotiate
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-Request-Id: f7c6281b-a467-4c4a-a46c-d23e10ba3535
< X-Runtime: 0.033282
< X-Ua-Compatible: IE=edge
< X-Xss-Protection: 1; mode=block
<
* Ignoring the response-body
* Connection #1 to host gitlab.domain-name.local left intact
* Issue another request to this URL: 'https://gitlab.domain-name.local/sa-apps/repository-name.git/git-upload-pack'
* Found bundle for host gitlab.domain-name.local: 0x31335b0 [can pipeline]
* Could pipeline, but not asked to!
* Re-using existing connection! (#1) with host gitlab.domain-name.local
* Connected to gitlab.domain-name.local (10.NN.NN.NN) port 443 (#1)
* Server auth using Negotiate with user ''
> POST /sa-apps/repository-name.git/git-upload-pack HTTP/1.1
Host: gitlab.domain-name.local
Authorization: Negotiate YIIG/gY[...snip...]
User-Agent: git/2.20.1.windows.1
Accept-Encoding: deflate, gzip
Content-Type: application/x-git-upload-pack-request
Accept: application/x-git-upload-pack-result
Content-Length: 0

Oops. Content-Length: is now 0!

* schannel: client wants to read 16384 bytes
* schannel: encrypted data buffer: offset 0 length 17408
* schannel: encrypted data got 517
* schannel: encrypted data buffer: offset 517 length 17408
* schannel: decrypted data length: 488
* schannel: decrypted data added: 488
* schannel: decrypted data cached: offset 488 length 16384
* schannel: encrypted data buffer: offset 0 length 17408
* schannel: decrypted data buffer: offset 488 length 16384
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 488
* schannel: decrypted data buffer: offset 0 length 16384
* The requested URL returned error: 500 Internal Server Error
* Closing connection 1
* schannel: shutting down SSL/TLS connection with gitlab.domain-name.local port 443
* schannel: clear security context handle
error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500 Internal Server Error
fatal: the remote end hung up unexpectedly

Not surprising. We sent an empty request.

With GfW 2.20.0, the Content-Length: is set correctly in the authenticated leg of the POST request. At the same point in the log, I see

User-Agent: git/2.20.0.windows.1
Accept-Encoding: deflate, gzip
Content-Type: application/x-git-upload-pack-request
Accept: application/x-git-upload-pack-result
Content-Length: 625

* upload completely sent off: 625 out of 625 bytes
* schannel: client wants to read 16384 bytes
* schannel: encrypted data buffer: offset 0 length 17408
* schannel: encrypted data got 4162
* schannel: encrypted data buffer: offset 4162 length 17408
* schannel: decrypted data length: 4133
* schannel: decrypted data added: 4133
* schannel: decrypted data cached: offset 4133 length 16384
* schannel: encrypted data buffer: offset 0 length 17408
* schannel: decrypted data buffer: offset 4133 length 16384
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 4133
* schannel: decrypted data buffer: offset 0 length 16384
< HTTP/1.1 200 OK
< Server: nginx
< Date: Wed, 16 Jan 2019 04:28:41 GMT
< Content-Type: application/x-git-upload-pack-result
< Transfer-Encoding: chunked
< Connection: keep-alive
< Cache-Control: no-cache
< Www-Authenticate: Negotiate oYG2MIG[...snip...]
< Strict-Transport-Security: max-age=31536000
<
20:28:41.896132 pkt-line.c:80           packet:   fetch-pack< NAK
20:28:41.896132 pkt-line.c:80           packet:   fetch-pack> 0000
20:28:41.897132 pkt-line.c:80           packet:     sideband< \2Enumerating objects: 952, done.
remote: Enumerating objects: 952, done.

etc. The fetch then succeeds.

#2012 likely reports the same bug. The same error is apparent in the log file gitOutput_redacted.txt attached to that issue (see lines 109 and 134).

[dscho]

The package difference between v2.20.0 and v2.20.1 can be seen by calling sdk cd build-extra && git diff --no-index ./versions/package-versions-2.20.0.txt ./versions/package-versions-2.20.1.txt in Git for Windows' SDK:

diff --git a/./versions/package-versions-2.20.0.txt b/./versions/package-versions-2.20.1.txt
index 86123906b..ad61304a2 100644
--- a/./versions/package-versions-2.20.0.txt
+++ b/./versions/package-versions-2.20.1.txt
@@ -11,7 +11,7 @@ expat 2.2.6-1
 file 5.35-1
 findutils 4.6.0-1
 gawk 4.2.1-1
-gcc-libs 7.3.0-3
+gcc-libs 7.4.0-1
 gettext 0.19.8.1-1
 git-extra 1.1.327.10b58d8-1
 git-flow 1.11.0-1
@@ -24,10 +24,10 @@ heimdal-libs 7.5.0-3
 icu 62.1-1
 less 530-1
 libasprintf 0.19.8.1-1
-libassuan 2.5.1-1
+libassuan 2.5.2-1
 libbz2 1.0.6-2
 libcrypt 2.1-2
-libcurl 7.62.0-3
+libcurl 7.63.0-1
 libedit 3.1-20170329
 libexpat 2.2.6-1
 libffi 3.2.1-3
@@ -51,10 +51,10 @@ libpcre 8.42-1
 libpcre2_8 10.32-1
 libpsl 0.20.2-1
 libreadline 7.0.005-1
-libsasl 2.1.27rc8-1
-libserf 1.3.9-2
+libsasl 2.1.27-1
+libserf 1.3.9-3
 libsqlite 3.21.0-4
-libssh2 1.8.0-1
+libssh2 1.8.0-2
 libtasn1 4.13-1
 libunistring 0.9.10-1
 libutil-linux 2.32.1-1
@@ -65,13 +65,13 @@ mingw-w64-x86_64-bzip2 1.0.6-6
 mingw-w64-x86_64-ca-certificates 20180409-1
 mingw-w64-x86_64-c-ares 1.15.0-1
 mingw-w64-x86_64-connect 1.105-1
-mingw-w64-x86_64-curl 7.62.0-3
+mingw-w64-x86_64-curl 7.63.0-1
 mingw-w64-x86_64-expat 2.2.6-1
 mingw-w64-x86_64-gcc-libs 7.3.0-2
 mingw-w64-x86_64-gettext 0.19.8.1-7
-mingw-w64-x86_64-git 2.20.0.1.9515583416-1
+mingw-w64-x86_64-git 2.20.1.1.7c9fbc07db-1
 mingw-w64-x86_64-git-credential-manager 1.18.3-1
-mingw-w64-x86_64-git-doc-html 2.20.0.1.9515583416-1
+mingw-w64-x86_64-git-doc-html 2.20.1.1.7c9fbc07db-1
 mingw-w64-x86_64-git-lfs 2.6.1-1
 mingw-w64-x86_64-gmp 6.1.2-1
 mingw-w64-x86_64-gnutls 3.6.5-1
@@ -91,13 +91,13 @@ mingw-w64-x86_64-libzip 1.5.1-1
 mingw-w64-x86_64-mpc 1.1.0-1
 mingw-w64-x86_64-mpfr 4.0.1-2
 mingw-w64-x86_64-nettle 3.4.1-1
-mingw-w64-x86_64-nghttp2 1.35.0-1
+mingw-w64-x86_64-nghttp2 1.35.1-1
 mingw-w64-x86_64-odt2txt 0.5-2
 mingw-w64-x86_64-openssl 1.1.1.a-1
 mingw-w64-x86_64-pcre 8.42-1
 mingw-w64-x86_64-pcre2 10.23-1
-mingw-w64-x86_64-tcl 8.6.8-1
-mingw-w64-x86_64-tk 8.6.8-1
+mingw-w64-x86_64-tcl 8.6.9-2
+mingw-w64-x86_64-tk 8.6.9.1-1
 mingw-w64-x86_64-wineditline 2.205-1
 mingw-w64-x86_64-wintoast 1.0.0.181.9b0663d-1
 mingw-w64-x86_64-xz 5.2.4-1

The one thing that sticks out is the mingw-w64-x86_64-curl change. Could you download both versions, as Portable Git, and then copy v2.20.0's mingw64\bin\libcurl-4.dll over v2.20.1's and then try in the latter whether that works around it?

[georgyo]

I can confirm that this problem is not windows specific. Git 2.20.1 on linux also breaks when doing SPNEGO, but only if libcurl has been compiled with http2 support. The same version of libcurl on 2.18 does not have this problem.

[dscho]

Interesting. Can you bisect further? The commit graph distance between 2.18 and 2.20.1 is rather large.

[georgyo]

I was about to start this, however the problem goes away in 2.21.0.

[dscho]

Really...? that's fantastic. Can you also verify that it is solved in the latest snapshot (which will hopefully be identical to Git for Windows v2.21.0 modulo a Git LFS update that is blocking that release)?

[kkm000]

Looks like the latest snapshot works.

2.20.1:

/$ git --version
git version 2.20.1.windows.1
cpu: x86_64
built from commit: 7c9fbc07db0e2939b36095df45864b8cda19b64f
sizeof-long: 4
sizeof-size_t: 8

/$ git --exec-path
C:/Users/kkm/Downloads/PortableGit-2.20.1-64-bit.7z/mingw64/libexec/git-core

/$ git clone https://gitlab/group/repo.git
Cloning into 'repo'...
error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500 Internal Server Error
fatal: the remote end hung up unexpectedly

Portable snapshot from https://wingit.blob.core.windows.net/files/PortableGit-prerelease-2.21.0.rc2.windows.1.707.gf6431ebcae-64-bit.7z.exe:

/$ git --version
git version 2.21.0.rc2.windows.1.707.gf6431ebcae
cpu: x86_64
built from commit: f6431ebcae0cbb476d9f565bbcfcbb463c421dc7
sizeof-long: 4
sizeof-size_t: 8

/$ git --exec-path
C:/Users/kkm/Downloads/PortableGit-2.21.0.rc2.1.707.gf6431ebcae/mingw64/libexec/git-core

/$ git clone https://gitlab/group/repo.git
Cloning into 'repo'...
remote: Enumerating objects: 16547, done.
remote: Counting objects: 100% (16547/16547), done.
remote: Compressing objects: 100% (5845/5845), done.
remote: Total 16547 (delta 7747), reused 16416 (delta 7639)
Receiving objects: 100% (16547/16547), 2.46 MiB | 26.26 MiB/s, done.
Resolving deltas: 100% (7747/7747), done.

/$ cd repo

/repo (master)$ git fsck --strict
Checking object directories: 100% (256/256), done.
Checking objects: 100% (16547/16547), done.

Phew! Looks like we are on a solid ground again! :-)

@georgyo, thanks for the heads-up. I'll watch for the broken version on Linux, too. I'd also appreciate if you double-confirm after me that the issue has been fixed. I'm not paranoid, I just have been using computers for quite a while...

[dscho]

Phew! Looks like we are on a solid ground again!

Excellent!