Skip to content

Merge error (Bitbucket) when using Git 2.27 - works with Git 2.24 #2682

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
1 task done
patrick-compass opened this issue Jun 10, 2020 · 17 comments
Closed
1 task done

Merge error (Bitbucket) when using Git 2.27 - works with Git 2.24 #2682

patrick-compass opened this issue Jun 10, 2020 · 17 comments

Comments

@patrick-compass
Copy link

patrick-compass commented Jun 10, 2020 •
edited
Loading

  • I was not able to find an open or closed issue matching what I'm seeing

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
$ git --version --build-options

**Working version**
git version 2.24.0.windows.2
cpu: x86_64
built from commit: 02af2cc5bfc37d4500e8a09333269e57f442198a
sizeof-long: 4
sizeof-size_t: 8

**Failing version**
version 2.27 64-bit
can't install due to live server
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
$ cmd.exe /c ver

Microsoft Windows [Version 10.0.14393]
  • What options did you set as part of the installation? Or did you choose the
    defaults?
# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

Editor Option: VIM
Custom Editor Path: 
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Disabled
Enable Builtin Interactive Add: Disabled
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

Manifested in Bitbucket pull requests. Error trying to display the merge diff. Attempt to merge produced this error.
image

Note: The error is definitely concerned with the repo containing a file named AUX which is invalid in Windows

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Testing with Bash, errors occuring within Bitbucket

I'm not sure what commands Bitbucket runs trying to render a pull request.
Error above shows failing command when attempting to merge anyway.
  • What did you expect to occur after running these commands?

When using 2.24 the pull request renders correctly and we're able to merge.

  • What actually happened instead?

When using 2.27 the pull request shows an error and trying to merge produces the error inserted above related to the invalid Windows filename "AUX"

  • If the problem was occurring with a specific repository, can you provide the
    URL to that repository to help us with testing?

working on private on-premise copy of linux kernel
specifically files like https://github.com/torvalds/linux/blob/master/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/aux.c
@derrickstolee
Copy link

Unfortunately, the Linux kernel repo uses paths that are not available on Windows. ("aux.*" is forbidden due to DOS compatibility. Let's...not get into that too much.)

The issue you are having is related to the security work introduced in v2.24.1, I believe. In order to work with such paths you will need to disable the core.protectNTFS setting. BE ADVISED THAT THERE ARE SECURITY IMPLICATIONS TO DISABLING THIS SETTING. Only apply this setting to repositories where it is required, never globally. Further, only fetch from trusted remotes.

Even worse, you might have problems when these paths are trying to be written. When I use the Linux kernel repo on Windows, I write the following into .git/info/sparse-checkout:

!*/aux\.*

then run git sparse-checkout init followed by git reset --hard to make sure everything is in a good state.

@patrick-compass
Copy link
Author

@derrickstolee Agreed, let's not talk about super-legacy device files...

I understand how to apply those settings on my workstation, but any idea how to force Bitbucket to do the same?

@derrickstolee
Copy link

I understand how to apply those settings on my workstation, but any idea how to force Bitbucket to do the same?

If this is an on-prem server, then maybe you can do the same -- assuming you can find the repo on-disk somewhere. You'll need to reach out to BitBucket support to be sure you don't break anything.

The only person at Atlassian I know is Bryan Turner, who is probably @bturner here on GitHub. Maybe they can help?

@patrick-compass
Copy link
Author

We're also going to be doing a Bitbucket upgrade soon. Hopefully this was addressed on Atlassian's side.

@bturner
Copy link

bturner commented Jun 10, 2020 •
edited
Loading

Hello all. Thanks for the mention, @derrickstolee.

"Addressed" is a difficult thing, when it comes to this change. We're aware of it, yes, and I invested a significant amount of time into experimenting with different options. But we're pretty much stuck between a rock and a hard place on it:

  • If we "just" disable core.protectNTFS everywhere when the system is running on Windows, then we're actively defeating security enhancements that have been made in response to vulnerabilities
  • If we "just" leave it enabled... Well, you know what happens because you're living it
  • There are other possibilities, of course, and they come with their own pitfalls

Prior to the way the restriction was enforced changing in Git for Windows 2.25 (pull request here), we were investigating approaches for disabling core.protectNTFS because the 2.24.1 restriction was too draconian; being unable to push a repository to a server hosted on Windows simply because it had ever contained a restricted path was too restrictive. With that change applied, however, we decided to stick with the defaults.

If you're upgrading to Bitbucket Server 7.x (any version), then you'll also pick up the change that replaces 3-way diffs (merge-based diffs) for pull requests with 2-way diffs (common ancestor-based diffs). At that point the series of operations that are resulting in the failure you're seeing will no longer run--except when actually merging a pull request. If the merge for the diff was failing, chances are the "real" merge will fail too. Based on that, I suspect no upgrade will (presently) help with this.

Can you confirm, does the pull request in question actually modify a restricted path? Probably the best way to proceed with this issue is to create a support request so our supporters (and likely myself) can work with you directly to gather details about your situation and help you resolve it.

(Lastly, I'll note that if you're still running a version of Bitbucket Server that uses 3-way diffs for pull requests, it means you must be on 6.10 or older. There are no 6.x releases which support running Git 2.27 on the server; 2.24 is the latest supported version. In general you likely won't have problems, since Git is generally highly backward-compatible, but you may. We're constantly making small (and sometimes large or very large) tweaks to Bitbucket Server to address changes in newer Git versions, so upgrading the server with a version it doesn't support can cause problems.)

Best regards,
Bryan Turner
Atlassian Bitbucket

@patrick-compass
Copy link
Author

I'll report back in after our upgrade.
Thanks Bryan

@patrick-compass
Copy link
Author

We updated to Bitbucket 7.3.1 which was supposed to support git 2.27, however we've run into the same issue as before.

Is there something I can configure on my server to allow this to work?

@dscho
Copy link
Member

dscho commented Jun 30, 2020

Unfortunately, the Linux kernel repo uses paths that are not available on Windows. ("aux.*" is forbidden due to DOS compatibility. Let's...not get into that too much.)

The issue you are having is related to the security work introduced in v2.24.1, I believe. In order to work with such paths you will need to disable the core.protectNTFS setting. BE ADVISED THAT THERE ARE SECURITY IMPLICATIONS TO DISABLING THIS SETTING. Only apply this setting to repositories where it is required, never globally. Further, only fetch from trusted remotes.

Even worse, you might have problems when these paths are trying to be written. When I use the Linux kernel repo on Windows, I write the following into .git/info/sparse-checkout:

!*/aux\.*

then run git sparse-checkout init followed by git reset --hard to make sure everything is in a good state.

I think that this advice still holds.

@patrick-compass
Copy link
Author

Unfortunately, the Linux kernel repo uses paths that are not available on Windows. ("aux.*" is forbidden due to DOS compatibility. Let's...not get into that too much.)
The issue you are having is related to the security work introduced in v2.24.1, I believe. In order to work with such paths you will need to disable the core.protectNTFS setting. BE ADVISED THAT THERE ARE SECURITY IMPLICATIONS TO DISABLING THIS SETTING. Only apply this setting to repositories where it is required, never globally. Further, only fetch from trusted remotes.
Even worse, you might have problems when these paths are trying to be written. When I use the Linux kernel repo on Windows, I write the following into .git/info/sparse-checkout:

!*/aux\.*

then run git sparse-checkout init followed by git reset --hard to make sure everything is in a good state.

I think that this advice still holds.

If it still applies, how do I configure this on the server? As far as I know, .gitconfig files are for the client side of the equation.

@patrick-compass
Copy link
Author

patrick-compass commented Jun 30, 2020 •
edited
Loading

Can you confirm, does the pull request in question actually modify a restricted path? Probably the best way to proceed with this issue is to create a support request so our supporters (and likely myself) can work with you directly to gather details about your situation and help you resolve it.

@bturner No, the PR in question doesn't even try to modify this file.

@mfriedrich74
Copy link

mfriedrich74 commented Jun 30, 2020 via email

@mfriedrich74
Copy link

mfriedrich74 commented Jun 30, 2020 via email

@bturner
Copy link

bturner commented Jun 30, 2020 •
edited
Loading

@lpd-patrick,

I'm not too surprised a Bitbucket Server upgrade didn't help. I tried to note up front that it wouldn't: "If the merge for the diff was failing, chances are the "real" merge will fail too. Based on that, I suspect no upgrade will (presently) help with this." 7.3.1 will support Git 2.27, but this issue isn't really about supporting Git 2.27; it's about a restriction that was added in Git 2.24.1 that, as I tried to explain above, we're basically stuck between a rock and a hard place on how to handle. Given we don't have any "perfect" options we've elected to go with what the Git maintainers have chosen and leave the protections enabled.

At this point, my suggestion would be, as @mfriedrich74 suggested, that you open a support case with Atlassian. There are a few different options we can investigate. The "easiest" would be to simply downgrade Git on your server. Of course, you lose all the bugfixes and other improvements that come with newer versions, but it also "disables" the core.protectNTFS checks. Of course, there's functionally no difference between that approach and simply adding core.protectNTFS=false to your global .gitconfig for the user your Bitbucket Server install runs as. As @derrickstolee and @dscho indicated, though, that has some security implications; whether those are acceptable to you and your team is for you to decide.

There are other options we can investigate that would make the workaround a bit more targeted, but a support case with Atlassian seems like the correct place to do that, rather than here on the Git for Windows issue board. I don't think there's a lot the Git for Windows developers can do to "fix" this; I'd imagine they're in a similar bind to us, in that having the safeguards on by default protects more users at the cost of inconveniencing some and having them off by default is more convenient but protects no one.

Best regards,
Bryan Turner
Atlassian Bitbucket

@patrick-compass
Copy link
Author

@bturner Thanks for checking back in. I'm not surprised either and I did open a support ticket this morning.

@bturner
Copy link

bturner commented Jun 30, 2020

Thanks @lpd-patrick. If you mentioned this issue on the support case, I'm sure I'll get looped in on the Atlassian side to help with things.

@patrick-compass
Copy link
Author

Indeed I did

@dscho
Copy link
Member

dscho commented Jul 1, 2020

Okay, let's close the ticket here.

@dscho dscho closed this as completed Jul 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@bturner @dscho @derrickstolee @mfriedrich74 @patrick-compass