Segmentation fault on cherry-pick

[liayn]

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options
git version 2.10.2.windows.1
sizeof-long: 4
machine: x86_64

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver
Microsoft Windows [Version 10.0.14393]

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt
Path Option: Cmd
Plink Path: C:\Program Files\TortoiseGit\bin\TortoiseGitPlink.exe
SSH Option: Plink
CRLF Option: CRLFCommitAsIs
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled

• Additional info

The repository is the TYPO3 CMS repository git://git.typo3.org/Packages/TYPO3.CMS.git
I run multiple worktrees for master/7-6/6-2/.

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Bash

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

# I'm on master branch in sync with origin/master
git fetch https://review.typo3.org/Packages/TYPO3.CMS refs/changes/60/50460/4
git cherry-pick FETCH_HEAD

• What did you expect to occur after running these commands?

The commit fetched from Gerrit should be applied on top.

• What actually happened instead?

Segmentation fault

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

A colleague tried the same command and it works.
I tried creating another "master" branch, but cherry-picking segfaults as well.
Cherry-picking onto another branch (different worktree) works as expected.
I run git gc git fsck ... git prune... and so on, nothing helped.

I did a complete new clone of the repo and applying the patch did work, but I actually do not want to redo my whole setup on a new checkout and I want to help finding the reason for a segfault.

Any help really appreciated. I could also provide my whole local repository for testing purposes, nothing confidential involved.

[liayn]

I now rebased the change on Gerrit directly (now set 5) and all of a sudden the patch could be applied again.

I'll still keep this report open, since no matter what happened here, a segmentation fault must never happen.

[dscho]

I seriously hope you kept a .zip of the directory tree. Because if you did not, we have exactly no chance to reproduce this issue, like, ever.

[liayn]

No, I did not, but running the above cherry-pick does still segfault. ;-)
I prepared the zip now :-D

Shall I upload the file here, or should I send it somewhere? > 200MB

[dscho]

@liayn thanks for keeping the .zip around. Would you have a chance to share it with me via, say, https://instant.io/?

[liayn]

@dscho I'm on the road a bit, so "streaming" the file to you is a bit hard. I uploaded it to our webserver now. Can I send you the link in a PM somehow?

[dscho]

Can I send you the link in a PM somehow?

@liayn sure. I'm on https://gitter.im/git-for-windows/git, for example. Also, all of my commits are signed off using my email address.

[dscho]

For the record, I received the rather large .zip and tried to reproduce the issue, but failed (the cherry-pick worked without segfaulting in my hands).

[liayn]

For the record: I did manage to respond to your mails earlier. Did so now. Let's see if we can still manage to see what's been happening. Thanks so far!

[dscho]

@liayn would it be possible to install the Git for Windows SDK and try to run the cherry-pick through gdb?

[liayn]

Sure, will give it a shot tomorrow.

[liayn]

Okay, I followed https://github.com/git-for-windows/git/wiki/Debugging-Git and kicked the optimization. I skipped step 3, since this seems to be governed by a CFLAGS check already.
Running make then fails for me though:

Markus@klein-lap MSYS /usr/src/git (master)
$ make
GIT_VERSION = 2.11.0.rc3.windows.1.dirty
    * new build flags
    CC credential-store.o
/bin/sh: cc: command not found
make: *** [Makefile:1995: credential-store.o] Error 127

@dscho Any suggestion for me?

[liayn]

Okay... one should update the wiki and should mention that I should not use the msys2_shell.cmd but simply open git-bash.exe :-(
ref https://github.com/git-for-windows/git/wiki/Technical-overview#installing-the-sdk

build is running now

[liayn]

here we go:

(gdb) r
Starting program: S:\git-sdk-64\mingw64\bin\git.exe "--exec-path=S:/git-sdk-64/usr/src/git" cherry-pick FETCH_HEAD
[New Thread 11508.0x3cb4]
[New Thread 11508.0x2a54]
[New Thread 11508.0x1a64]
[New Thread 11508.0x2d50]
[New Thread 11508.0x1980]
[New Thread 11508.0x1534]
[New Thread 11508.0x1274]
[New Thread 11508.0x22c4]
[New Thread 11508.0x1b80]
[New Thread 11508.0x468]
[New Thread 11508.0x3b4c]
[New Thread 11508.0x32f4]
[New Thread 11508.0x2478]
[New Thread 11508.0xf80]
[New Thread 11508.0x2058]
[New Thread 11508.0x3fac]
[New Thread 11508.0x940]
[New Thread 11508.0x3cd0]
[New Thread 11508.0x2e68]
[New Thread 11508.0xa8c]
[Thread 11508.0x22c4 exited with code 0]
[Thread 11508.0x1b80 exited with code 0]
[Thread 11508.0x1274 exited with code 0]
[Thread 11508.0x3b4c exited with code 0]
[Thread 11508.0x2e68 exited with code 0]
[Thread 11508.0xa8c exited with code 0]
[Thread 11508.0x1980 exited with code 0]
[Thread 11508.0x32f4 exited with code 0]
[Thread 11508.0x940 exited with code 0]
[Thread 11508.0x1534 exited with code 0]
[Thread 11508.0xf80 exited with code 0]
[Thread 11508.0x2478 exited with code 0]
[Thread 11508.0x468 exited with code 0]
[Thread 11508.0x2058 exited with code 0]
[Thread 11508.0x3cd0 exited with code 0]
[Thread 11508.0x3fac exited with code 0]

Thread 1 received signal SIGSEGV, Segmentation fault.
0x0000000000b51f18 in ?? ()
(gdb) back
#0  0x0000000000b51f18 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)

guess you need some more info, right?

[dscho]

one should update the wiki

I did that now. For future record: please just edit the wiki next time you see something wrong there. It's a wiki for that very reason: so that you (and everybody else) do not need to wait for me to do everything.

guess you need some more info, right?

Yes.

You may see different threads using the info threads command, then switch to another thread via, say, thread 1 and bt would then list the stack trace of that current thread.

If that does not help shed light into the issue, it would be best to single-step, or a variation thereof. This is how I would go about it:

1. set a breakpoint on cherry-pick's main function: b cmd_cherry_pick
2. start the debugging session: r
3. once execution stops in the main function, I would try to determine a reasonable next breakpoint by looking at the source: l (subsequent l commands will list more, l <lineno> will list from a given line number
4. then I would set another breakpoint and continue: b <lineno> and then c
5. If the crash occurs in between those breakpoints, I would try to figure out a break point between the last two, i.e. go back to step 3 above (with the difference that the debugging session has to be restarted with r instead of continued with c).
6. If there is actually no line between the two latest breakpoints, i.e. if the crash occurs in a function that was called from the line on which rests the second-latest breakpoint, I would restart with r and then step into that function using s. Then repeat as above.
7. If the crash does not occur between those two breakpoints, I would then disable the first breakpoint (so that subsequent restarts of the debugging session will not stop unnecessarily): dis <number> (where <number> is the breakpoint's number that was printed when you set it via b <function-or-line>.

This will need a little bit of tenacity, but should get you consistently to the culprit.