1.2.0 Cache permission issues #6400
Comments
|
Are you using the borg "pyinstaller-made fat binary" from the github releases page? |
|
|
BTW, it is a bit unusual what you are doing. The cache is usually a local cache used by one machine and one user. So, if you move that to a network share on a server and use it by multiple machines, you must trust them all. So, there are potential problems lurking in case something goes wrong and you trusted too much. |
|
Your reproducer script is doing way too much and requires root, how about this: Updated: ls needs |
|
One potential issue in your usage is inconsistent usage of If you use that borg option, you always need to use it in a consistent way. |
|
Output of the above script: |
|
Hmm, guess the unexpected modes of So guess we need a Not sure why some of the directory modes are also wrong. |
|
The file mode issue was introduced by the race-condition fix that introduced The directory mode issue is likely present since longer, but most users did not notice, because they didn't need group or other permissions. |
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Mar 4, 2022
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Mar 4, 2022
…gbackup#6400 we tried to be very private / secure here, but that created the issue that a less secure umask (like e.g. 0o007) just did not work. to make the umask work, we must start from 0o777 mode and let the umask do its work, like e.g. 0o777 & ~0o007 --> 0o770. with borg's default umask of 0o077, it usually ends up being 0o700, so only permissions for the user (not group, not others).
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Mar 4, 2022
…up#6400 we tried to be very private / secure here, but that created the issue that a less secure umask (like e.g. 0o007) just did not work. to make the umask work, we must start from 0o777 mode and let the umask do its work, like e.g. 0o777 & ~0o007 --> 0o770. with borg's default umask of 0o077, it usually ends up being 0o700, so only permissions for the user (not group, not others).
|
After applying PR #6403 it looks like this: |
|
Also, for the usual |
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Mar 9, 2022
…up#6400 we tried to be very private / secure here, but that created the issue that a less secure umask (like e.g. 0o007) just did not work. to make the umask work, we must start from 0o777 mode and let the umask do its work, like e.g. 0o777 & ~0o007 --> 0o770. with borg's default umask of 0o077, it usually ends up being 0o700, so only permissions for the user (not group, not others).
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Mar 9, 2022
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Mar 9, 2022
…up#6400 we tried to be very private / secure here, but that created the issue that a less secure umask (like e.g. 0o007) just did not work. to make the umask work, we must start from 0o777 mode and let the umask do its work, like e.g. 0o777 & ~0o007 --> 0o770. with borg's default umask of 0o077, it usually ends up being 0o700, so only permissions for the user (not group, not others).
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Aug 4, 2022
…orgbackup#6400 hopefully this is the final fix. after first fixing of borgbackup#6400 (by using os.umask after mkstemp), there was a new problem that chmod was not supported on some fs. even after fixing that, there were other issues, see the ACLs issue documented in borgbackup#6933. the root cause of all this is tempfile.mkstemp internally using a very secure, but hardcoded and for our use case problematic mode of 0o600. mkstemp_mode (mosty copy&paste from python stdlib tempfile module + "black" formatting applied) supports giving the mode via the api, that is the only change needed. slightly dirty due to the _xxx imports from tempfile, but hopefully this will be supported in some future python version.
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Aug 4, 2022
…orgbackup#6400 hopefully this is the final fix. after first fixing of borgbackup#6400 (by using os.umask after mkstemp), there was a new problem that chmod was not supported on some fs. even after fixing that, there were other issues, see the ACLs issue documented in borgbackup#6933. the root cause of all this is tempfile.mkstemp internally using a very secure, but hardcoded and for our use case problematic mode of 0o600. mkstemp_mode (mosty copy&paste from python stdlib tempfile module + "black" formatting applied) supports giving the mode via the api, that is the only change needed. slightly dirty due to the _xxx imports from tempfile, but hopefully this will be supported in some future python version.
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Aug 4, 2022
…orgbackup#6400 hopefully this is the final fix. after first fixing of borgbackup#6400 (by using os.umask after mkstemp), there was a new problem that chmod was not supported on some fs. even after fixing that, there were other issues, see the ACLs issue documented in borgbackup#6933. the root cause of all this is tempfile.mkstemp internally using a very secure, but hardcoded and for our use case problematic mode of 0o600. mkstemp_mode (mosty copy&paste from python stdlib tempfile module + "black" formatting applied) supports giving the mode via the api, that is the only change needed. slightly dirty due to the _xxx imports from tempfile, but hopefully this will be supported in some future python version.
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Aug 4, 2022
…orgbackup#6400 hopefully this is the final fix. after first fixing of borgbackup#6400 (by using os.umask after mkstemp), there was a new problem that chmod was not supported on some fs. even after fixing that, there were other issues, see the ACLs issue documented in borgbackup#6933. the root cause of all this is tempfile.mkstemp internally using a very secure, but hardcoded and for our use case problematic mode of 0o600. mkstemp_mode (mosty copy&paste from python stdlib tempfile module + "black" formatting applied) supports giving the mode via the api, that is the only change needed. slightly dirty due to the _xxx imports from tempfile, but hopefully this will be supported in some future python version.
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Aug 4, 2022
…orgbackup#6400 hopefully this is the final fix. after first fixing of borgbackup#6400 (by using os.umask after mkstemp), there was a new problem that chmod was not supported on some fs. even after fixing that, there were other issues, see the ACLs issue documented in borgbackup#6933. the root cause of all this is tempfile.mkstemp internally using a very secure, but hardcoded and for our use case problematic mode of 0o600. mkstemp_mode (mosty copy&paste from python stdlib tempfile module + "black" formatting applied) supports giving the mode via the api, that is the only change needed. slightly dirty due to the _xxx imports from tempfile, but hopefully this will be supported in some future python version.
This was referenced Aug 4, 2022
ThomasWaldmann
added a commit
to ThomasWaldmann/borg
that referenced
this issue
Aug 5, 2022
…orgbackup#6400 hopefully this is the final fix. after first fixing of borgbackup#6400 (by using os.umask after mkstemp), there was a new problem that chmod was not supported on some fs. even after fixing that, there were other issues, see the ACLs issue documented in borgbackup#6933. the root cause of all this is tempfile.mkstemp internally using a very secure, but hardcoded and for our use case problematic mode of 0o600. mkstemp_mode (mosty copy&paste from python stdlib tempfile module + "black" formatting applied) supports giving the mode via the api, that is the only change needed. slightly dirty due to the _xxx imports from tempfile, but hopefully this will be supported in some future python version.
pytorchmergebot
pushed a commit
to pytorch/pytorch
that referenced
this issue
Aug 24, 2023
### Description The `download_url_to_file` function in torch.hub uses a temporary file to prevent overriding a local working checkpoint with a broken download.This temporary file is created using `NamedTemporaryFile`. However, since `NamedTemporaryFile` creates files with overly restrictive permissions (0600), the resulting download will not have default permissions and will not respect umask on Linux (since moving the file will retain the restrictive permissions of the temporary file). This is especially problematic when trying to share model checkpoints between multiple users as other users will not even have read access to the file. The change in this PR fixes the issue by using custom code to create the temporary file without changing the permissions to 0600 (unfortunately there is no way to override the permissions behaviour of existing Python standard library code). This ensures that the downloaded checkpoint file correctly have the default permissions applied. If a user wants to apply more restrictive permissions, they can do so via usual means (i.e. by setting umask). See these similar issues in other projects for even more context: * borgbackup/borg#6400 * borgbackup/borg#6933 * zarr-developers/zarr-python#325 ### Issue #81297 ### Testing Extended the unit test `test_download_url_to_file` to also check permissions. Pull Request resolved: #82869 Approved by: https://github.com/vmoens
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Have you checked borgbackup docs, FAQ, and open Github issues?
Yes
Is this a BUG / ISSUE report or a QUESTION?
BUG/ISSUE
System information. For client/server mode post info for both machines.
Debian 11 as a Gen2 Hyper-V VM.
Your borg version (borg -V).
1.2.0 (does not work as intended) & 1.117 (works as intended)
Operating system (distribution) and version.
Debian 11 as a Gen2 Hyper-V VM. - 5.10.0-11-amd64
Hardware / network configuration, and filesystems used.
Filesystem: ext4
Describe the problem you're observing.
Borg 1.2.0 is setting permission of cache files to 700, no matter what --umask or "umask" in script is set to.
This causes problems when running borg on different computers using a single shared cache (network share), as only the owner will have permission, and not the group.
Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.
Yes, script:
Output:
The text was updated successfully, but these errors were encountered: