Can't clone the repository onto an NTFS drive

[eisenwave]

I can't clone the repository onto an NTFS drive.
Here's what happens:

Cloning into 'chap'...
remote: Enumerating objects: 30, done.
remote: Counting objects: 100% (30/30), done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 2507 (delta 9), reused 16 (delta 7), pack-reused 2477
Receiving objects: 100% (2507/2507), 52.77 MiB | 17.40 MiB/s, done.
Resolving deltas: 100% (1628/1628), done.
error: unable to create file test/expectedOutput/ELF32/LibcMalloc/LongStringTest/core.26548.list_used::extend:->%LongString=>StopHere::minoutgoing:%LongString=1: Invalid argument
error: unable to create file test/expectedOutput/ELF32/LibcMalloc/LongStringTest/core.26548.list_used::minoutgoing:%LongString=1: Invalid argument
error: unable to create file test/expectedOutput/ELF64/LibcMalloc/HasContainersAndSymbols/core.38066.count_used::extend:->: Invalid argument
error: unable to create file test/expectedOutput/ELF64/LibcMalloc/HasContainersAndSymbols/core.38066.count_used::extend:
...

This happens because of special characters like > in filenames, which are reserved.

[timboddy]

Can you clarify why you think support for NTFS drives matters?

In general chap is intended to be run on Linux, and the special characters you mention arrive automatically due to the way the file names are generated when redirect is on.

[eisenwave]

It's possible to have NTFS drives on Linux too. I in particular encountered this issue when cloning it onto an NTFS drive where I store various repositories. It's an NTFS drive so that I can open it up in Windows too (dual booting).

The existence of an NTFS drive used by a Linux install is by no means unusual. Especially with more users than ever using WSL, I wouldn't be surprised if more people didn't encounter these problems.

[timboddy]

From my perspective, the fact that Linux can mount a Linux file system does not mean that there is a strong need to be able to install chap on one.

With that said, I am not totally against someone providing such support. I have no plans to do so myself but would not be unwilling to review a pull request for such a change, and might accept it provided that the change addressed concerns such as, but not necessarily limited to, the following:

The reason the special characters are present in names of files in the repository is that the test directories contain expected output files that were generated with "redirect" turned on. In such a case, rather than put output to standard output, chap constructs a file name based on the core file name followed by a "." followed by information about the command that was being run. What this means is that truly supporting NTFS file systems, if one were to want to actually run chap on them, would mean that the automatic construction of output file names would have to be changed accordingly.

The document cited in the original statement (https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file) has some restrictions that chap will never support. For example, the document states "Do not assume case sensitivity. " This is a problem because C++ most definitely assumes case sensitivity and "foo" is a different type name than "Foo" or "FOO" ....

This matters because a command like "list used foo" on core someCore would result in a generated file name like "someCore.list_used_foo", where as "list used Foo" on core someCore would cause a filename "someCore.list_used_Foo".

The quoted document states that ":" should not be used in a file name. This is problematic given the above file name generation because C++ uses "::" in fully qualified type names, like "NS1::Foo" so the following command will currently generate an illegal name if redirect is on.

list used NS1::Foo

The quoted document states that "<" and ">" should not be used in a file name. This is problematic because the extend switch uses "->" rather heavily "<-".

If we decide to support NTFS will mean that people who make changes to chap may break this support unless they also test on NTFS. It seems a bit of a burden to expect people who change chap to expect them to have to test to make sure that cloning on NTFS still works and that running the tests on NTFS will still work. So I guess whoever made the change would need to set up something like Jenkins to make sure that a proposed pull request would also work on NTFS, unless with stipulate that NTFS support may break at any time and that it is the responsibility of any NTFS users who notice such breakage to submit pull requests for fixes or to persuade someone else to do so.

[kiplingw]

For what it's worth, I'm not able to checkout on a btrfs filesystem. Similar error:

...
remote: Total 2957 (delta 264), reused 324 (delta 158), pack-reused 2477
Receiving objects: 100% (2957/2957), 54.41 MiB | 2.44 MiB/s, done.
Resolving deltas: 100% (1883/1883), done.
error: unable to create file test/expectedOutput/ELF64/LibcMalloc/HasContainersAndSymbols/core.38066.show_used_HasSet::commentExtensions:true::extend:HasSet@18->@0=>mapNode:mapNode@10->@0=>mapNode:mapNode@18->@0=>mapNode:mapNode@20->=>StopHere: File name too long
error: unable to create file test/expectedOutput/ELF64/LibcMalloc/MapOrSetPatternTest/core.59709.describe_used_%MapOrSetNode::commentExtensions:true::extend:%MapOrSetNode->%MapOrSetNode::maxincoming:%MapOrSetNode=0::skipUnfavoredReferences:true: File name too long
error: unable to create file test/expectedOutput/ELF64/LibcMalloc/UnorderedMapOrSetPatternTest/core.3522.describe_used_%UnorderedMapOrSetNode::commentExtensions:true::extend:%UnorderedMapOrSetNode->%UnorderedMapOrSetNode::maxincoming:%UnorderedMapOrSetNode=0::skipUnfavoredReferences:true: File name too long
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

[adamchainz]

on macOS with APFS I'm seeing similar warnings about case-sensitive paths:

git clone https://github.com/vmware/chap.git
Cloning into 'chap'...
cd remote: Enumerating objects: 3034, done.
remote: Counting objects: 100% (557/557), done.
remote: Compressing objects: 100% (333/333), done.
remote: Total 3034 (delta 324), reused 373 (delta 187), pack-reused 2477
Receiving objects: 100% (3034/3034), 54.43 MiB | 1.45 MiB/s, done.
Resolving deltas: 100% (1943/1943), done.
warning: the following paths have collided (e.g. case-sensitive paths
on a case-insensitive filesystem) and only one from the same
colliding group is in the working tree:

  'test/expectedOutput/ELF64/LibcMalloc/HasSymbols_CoreHasMangledTypeNames_NoSymdefs/core.34218.count_used_B'
  'test/expectedOutput/ELF64/LibcMalloc/HasSymbols_CoreHasMangledTypeNames_NoSymdefs/core.34218.count_used_b'
  'test/expectedOutput/ELF64/LibcMalloc/HasSymbols_CoreHasMangledTypeNames_NoSymdefs/core.34218.count_used_D'
  'test/expectedOutput/ELF64/LibcMalloc/HasSymbols_CoreHasMangledTypeNames_NoSymdefs/core.34218.count_used_d'

[timboddy]

With respect to the comment about btrfs, I am puzzled here. My impression is that btrfs supports file names of up to 255 characters, and the "File name too long" messages are on file names that are shorter than that.

With respect to the comment about APFS, isn't BSD available on maxOS? If so, that might be an alternative. My guess is that you will need to also add support for a different core file format and perhaps for a different malloc library.

[adamchainz]

With respect to the comment about APFS, isn't BSD available on maxOS? If so, that might be an alternative. My guess is that you will need to also add support for a different core file format and perhaps for a different malloc library.

I am using chaps to analyze core dumps from my linux servers, so I don't personally need any such support. I just noted the issue with the repo being incompatible with case-insensitive FS's.