Sparse-checkout excluded files show as deleted in git status after some operations

[LunNova]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? 32-bit or 64-bit? Include the
  output of git version as well.

64 bit git for windows 2.9

$ git --version
git version 2.9.0.windows.1

edit: Still present in 2.9.2.windows.1

• Which version of Windows are you running? 32-bit or 64-bit?

  64-bit Windows 7

• What options did you set as part of the installation? Or did you choose the
  defaults?

    filemode = false
    preloadindex = false
    fscache = false

Also tested with preloadindex and fscache, with no change in the results.

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

I don't think so.

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

  Doesn't matter, git bash or cmd makes no difference.

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

I have created an MCVE repository which can be checked out and used to replicate the issue.

Checkout the repository at: https://github.com/nallar/git-issue-sparse-checkout

git clone https://github.com/nallar/git-issue-sparse-checkout.git mcve-sparse-checkout
cd mcve-sparse-checkout
git checkout other
git checkout master
git config core.sparseCheckout true
cp sparse-checkout .git/info/sparse-checkout
git checkout -f master
git reset --hard HEAD
# We now have sparse checkout enabled, so the folder "excluded-folder" should not be present
ls
# We should have no changes so far
git status
# Cherry-pick top commit from the other branch
git cherry-pick other
# That cherry-pick succeeded with no conflicts. We should have no unstaged changes
git status
# Oh dear, now it shows 'deleted:    excluded-folder/1.txt'
# Workaround, set skip-worktree which for some reason got unset:
git update-index --skip-worktree excluded-folder/1.txt
# OR
git checkout -f master
# Now it's showing the correct result again
git status
# no changes

• What did you expect to occur after running these commands?

  Cherry-picking a commit from another branch should not cause files excluded in the sparse checkout rules to be changed.

• What actually happened instead?

  For some reason, the skip-worktree flag gets unset on the file excluded-folder/1.txt causing this to show as deleted. Note that the file which broke is a file which was renamed in a commit on the other branch, but was not renamed in the commit we actually cherry-picked.

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

https://github.com/nallar/git-issue-sparse-checkout

TL;DR

A file which is excluded by sparse-checkout will show up as deleted in git status after cherry-picking a commit from another branch if any prior commit in that branch renames the file.

[LunNova]

Is there any other information I can give to help debug this?

I don't know where to start trying to look into the cause. :(

[robertlagrant]

I think I had something like this with SourceTree, although you've debugged it much better than I ever did: some (not all) files I had previously excluded via sparse checkout ended up being deleted in commits I made long after they'd been excluded. As in, folders containing them were not on my file system, that's how excluded they were :-)

I don't know the exact version now; I'll check and add when I get to work.

[dscho]

I just tried to reproduce this (with a patch on top of v2.12.0-rc2, see git/git@master...dscho:sparse-cherry-pick for details) and it seems that the bug miraculously went away.

Could you verify my claim? (If you find that your original problem still persists, I would love to work with you to condense the MCVE into a test case in Git's regression test suite, which is the first step to get it fixed.)

[LunNova]

The issue persists. The original MCVE fails. Your simplified version does not run into the issue.

https://gist.github.com/nallar/bb4e5d654c41302eb1b14855b1ba96a1

[dscho]

In what little time I had to look at this problem, I got the impression that this issue is not Windows-specific at all. And as the Git project is blessed with a lot more active contributors than the Git for Windows project, I recommend sending this report to the Git mailing list instead. Hopefully one of the bunch will jump on this issue and help you fix it.

[LunNova]

I reported this to the mailing list on 6 Apr 2017 and unfortunately got no response.

[dscho]

@nallar sorry to hear that.

In the meantime, one of my excellent colleagues worked on sparse checkouts, and came up with a couple of fixes. In particular, I think about: microsoft@d7afc82, microsoft@fc0a289, microsoft@2121831 and microsoft@d55c451.

Could you maybe cherry-pick them into a local SDK and try whether any of them fix your problem? (I sadly am short on time, so it would be really helpful to try these patches and let me know whether they fix this ticket; I would then of course fast-track the relevant patches into Git for Windows.)

[TomyLobo]

I havent tested OP's repro steps, but on Ubuntu with Git 2.13.0 "git pull" on my sparse checkout still messes up and makes "git status" show that I deleted files that were not part of the sparse checkout in 2.13.0.
These files were added in the pull.
The changelogs for 2.13.1/2/3 do not contain the word "sparse", so I think this problem is still unfixed in upstream Git's latest release.

[dscho]

@TomyLobo did you cherry-pick any of the commits I indicated, to see whether they fix anything for you?

[TomyLobo]

@dscho That was more than 2 months old at the time, I had simply assumed they had already hit mainline, and anyway, since I am using Linux, not Windows, I'm not sure how useful that would be :)

I was merely mentioning it in order to highlight the fact that it's unfixed in upstream and maybe should be transported there, perhaps together with my comment, to indicate that there is an issue on Linux as well.

[dscho]

since I am using Linux, not Windows, I'm not sure how useful that would be :)

I think it would be highly useful, as nothing in those patches should be specific to Windows.

[dscho]

I was merely mentioning it in order to highlight the fact that it's unfixed in upstream and maybe should be transported there, perhaps together with my comment, to indicate that there is an issue on Linux as well.

Just to clarify: if you test this on Linux, and verify that it fixes things, you help bump up the priority of submitting that patch upstream. The more you help, the more you get what you want.

[dscho]

I just ran the MCVE again, and this seems to have been resolved in the meantime.