Fix the Azure Pipeline (e.g. macOS builds after the upgrade to Mojave) #137
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The git-p4 login ticket expiry test causes unreliable test runs. Since the handling of ticket expiry in git-p4 is far from polished anyway, let's remove it for now. A better way to actually run the test is to create a python "fake" version of "p4" which returns whatever expiry results the test requires. Ideally git-p4 would look at the expiry time before starting any long operations, and cleanup gracefully if there is not enough time left. But that's quite hard to do. Signed-off-by: Luke Diamand <[email protected]> Signed-off-by: Junio C Hamano <[email protected]>
The write_or_die() function has one quirk that a caller might not expect: when it sees EPIPE from the write() call, it translates that into a death by SIGPIPE. This doesn't change the overall behavior (the program exits either way), but it does potentially confuse test scripts looking for a non-signal exit code. Let's switch away from using write_or_die() in a few code paths, which will give us more consistent exit codes. It also gives us the opportunity to write more descriptive error messages, since we have context that write_or_die() does not. Note that this won't do much by itself, since we'd typically be killed by SIGPIPE before write_or_die() even gets a chance to do its thing. That will be addressed in the next patch. Signed-off-by: Jeff King <[email protected]> Signed-off-by: Junio C Hamano <[email protected]>
The default SIGPIPE behavior can be useful for a command that generates a lot of output: if the receiver of our output goes away, we'll be notified asynchronously to stop generating it (typically by killing the program). But for a command like fetch, which is primarily concerned with receiving data and writing it to disk, an unexpected SIGPIPE can be awkward. We're already checking the return value of all of our write() calls, and dying due to the signal takes away our chance to gracefully handle the error. On Linux, we wouldn't generally see SIGPIPE at all during fetch. If the other side of the network connection hangs up, we'll see ECONNRESET. But on OS X, we get a SIGPIPE, and the process is killed. This causes t5570 to racily fail, as we sometimes die by signal (instead of the expected die() call) when the server side hangs up. Let's ignore SIGPIPE during the network portion of the fetch, which will cause our write() to return EPIPE, giving us consistent behavior across platforms. This fixes the test flakiness, but note that it stops short of fixing the larger problem. The server side hit a fatal error, sent us an "ERR" packet, and then hung up. We notice the failure because we're trying to write to a closed socket. But by dying immediately, we never actually read the ERR packet and report its content to the user. This is a (racy) problem on all platforms. So this patch lays the groundwork from which that problem might be fixed consistently, but it doesn't actually fix it. Note the placement of the SIGPIPE handling. The absolute minimal change would be to ignore SIGPIPE only when we're writing. But twiddling the signal handler for each write call is inefficient and maintenance burden. On the opposite end of the spectrum, we could simply declare that fetch does not need SIGPIPE handling, since it doesn't generate a lot of output, and we could just ignore it at the start of cmd_fetch(). This patch takes a middle ground. It ignores SIGPIPE during the network operation (which is admittedly most of the program, since the actual network operations are all done under the hood by the transport code). So it's still pretty coarse. Signed-off-by: Jeff King <[email protected]> Signed-off-by: Junio C Hamano <[email protected]>
Due to reasons (some XCode versions seem to include gettext, some don't?), Homebrew does not expose the libraries and headers in /usr/local/ by default anymore. Let's help find them again. Note: for some reason, this is a change of behavior caused by the upgrade to Mojave, identified in our Azure Pipeline; it seems that Homebrew used to add the /usr/local/ directories to the include and link search path before, but now it no longer does. Signed-off-by: Johannes Schindelin <[email protected]> Signed-off-by: Junio C Hamano <[email protected]>
Most notably, it seems that macOS' APFS does not allow that. Signed-off-by: Johannes Schindelin <[email protected]> Signed-off-by: Junio C Hamano <[email protected]>
This is patch 1/2 that is needed to cope with Azure Pipelines' macOS agents' upgrade to Mojave (which uncovered two bugs in Git).
This is patch 2/2 that is needed to cope with Azure Pipelines' macOS agents' upgrade to Mojave (which uncovered two bugs in Git).
Recently the Git for Windows project started the upgrade process to a MSYS2 runtime version based on Cygwin v3.x. This has the very notable consequence that `$(uname -r)` no longer reports a version starting with "2", but a version with "3". That breaks our build, as df5218b (config.mak.uname: support MSys2, 2016-01-13) simply did not expect the version reported by `uname -r` to depend on the underlying Cygwin version: it expected the reported version to match the "2" in "MSYS2". So let's invert that test case to test for *anything else* than a version starting with "1" (for MSys). That should safeguard us for the future, even if Cygwin ends up releasing versionsl like 314.272.65536. Signed-off-by: Johannes Schindelin <[email protected]> Signed-off-by: Junio C Hamano <[email protected]>
This branch fixes the flakiness of the git-daemon tests on macOS.
dscho
changed the title
dscho
force-pushed
the
vfs-mojave-fix
branch
4 times, most recently
from
3739657 to
bf48727
Compare
This removes a test that failed frequently on Azure Pipelines' macOS agents, indicating some serious flakiness.
In Git for Windows, we use the MSYS2 Bash which inherits a non-standard PID model from Cygwin's POSIX emulation layer: every MSYS2 process has a regular Windows PID, and in addition it has an MSYS2 PID (which corresponds to a shadow process that emulates Unix-style signal handling). With the upgrade to the MSYS2 runtime v3.x, this shadow process cannot be accessed via `OpenProcess()` any longer, and therefore t6500 thought incorrectly that the process referenced in `gc.pid` (which is not actually a real `gc` process in this context, but the current shell) no longer exists. Let's fix this by making sure that the Windows PID is written into `gc.pid` in this test script soo that `git.exe` is able to understand that that process does indeed still exist. Signed-off-by: Johannes Schindelin <[email protected]>
This is a (partial) backport of the improvements of the Azure Pipeline made in Git for Windows v2.21.0. The major benefit is that it makes the task to initialize the Git for Windows SDK *a lot* faster. It would also have worked around a problem where an MSYS2 update breaks t6500.13, and it will work around similar problems in the future, as the Azure Pipeline that updates git-sdk-64-minimal will not publish a new artifact unless Git's test suite (as of upstream's current `master`) passes.
We will need this in the next commit, where we backport the parallelized tests on Windows.
It just takes forever, or at least over an hour, to run them in a non-parallel way. That's why we switched to parallel testing in v2.21.0, and hereby we bring that goodness back to vfs-2.20.1.
While working on parallelizing the tests in Azure Pipelines, an issue was discovered with the `is_msys2_sh()` function: it expects the path components to be separated by exactly one dir separator. That does not need to be the case, though, e.g. when the components in the `PATH` variable have trailing slashes. Let's make the code much more robust in this respect. This fixes t5580.5. Signed-off-by: Johannes Schindelin <[email protected]>
The tests on Windows arguably take the longest, so let's get the build done with, first thing.
GCC 8 introduced the new -Wcast-function-type warning, which is
implied by -Wextra (which, in turn is enabled in our DEVELOPER flags).
When building Git with GCC 8 and this warning enabled on a non-glibc
platform [1], one is greeted with a screenful of compiler
warnings/errors:
compat/obstack.c: In function '_obstack_begin':
compat/obstack.c:162:17: error: cast between incompatible function types from 'void * (*)(long int)' to 'struct _obstack_chunk * (*)(void *, long int)' [-Werror=cast-function-type]
h->chunkfun = (struct _obstack_chunk * (*)(void *, long)) chunkfun;
^
compat/obstack.c:163:16: error: cast between incompatible function types from 'void (*)(void *)' to 'void (*)(void *, struct _obstack_chunk *)' [-Werror=cast-function-type]
h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
^
compat/obstack.c:116:8: error: cast between incompatible function types from 'struct _obstack_chunk * (*)(void *, long int)' to 'struct _obstack_chunk * (*)(long int)' [-Werror=cast-function-type]
: (*(struct _obstack_chunk *(*) (long)) (h)->chunkfun) ((size)))
^
compat/obstack.c:168:22: note: in expansion of macro 'CALL_CHUNKFUN'
chunk = h->chunk = CALL_CHUNKFUN (h, h -> chunk_size);
^~~~~~~~~~~~~
<snip>
'struct obstack' stores pointers to two functions to allocate and free
"chunks", and depending on how obstack is used, these functions take
either one parameter (like standard malloc() and free() do; this is
how we use it in 'kwset.c') or two parameters. Presumably to reduce
memory footprint, a single field is used to store the function pointer
for both signatures, and then it's casted to the appropriate signature
when the function pointer is accessed. These casts between function
pointers with different number of parameters are what trigger those
compiler errors.
Modify 'struct obstack' to use unions to store function pointers with
different signatures, and then use the union member with the
appropriate signature when accessing these function pointers. This
eliminates the need for those casts, and thus avoids this compiler
error.
[1] Compiling 'compat/obstack.c' on a platform with glibc is sort of
a noop, see the comment before '# define ELIDE_CODE', so this is
not an issue on common Linux distros.
Signed-off-by: SZEDER Gábor <[email protected]>
The return type of the `GetProcAddress()` function is `FARPROC` which evaluates to `long long int (*)()`, i.e. it cannot be cast to the correct function signature by GCC 8. To work around that, we first cast to `void *` and go on with our merry lives. Signed-off-by: Johannes Schindelin <[email protected]>
The kwset functionality makes use of the obstack code, which expects to be handed a function that can allocate large chunks of data. It expects that function to accept a `size` parameter of type `long`. This upsets GCC 8 on Windows, because `long` does not have the same bit size as `size_t` there. Now, the proper thing to do would be to switch to `size_t`. But this would make us deviate from the "upstream" code even further, making it hard to synchronize with newer versions, and also it would be quite involved because that `long` type is so invasive in that code. Let's punt, and instead provide a super small wrapper around `xmalloc()`. Signed-off-by: Johannes Schindelin <[email protected]>
This is needed to make things compile with GCC 8.x and later. Signed-off-by: Johannes Schindelin <[email protected]>
This backports several patch series from Git for Windows' `master` to make things compile with GCC v8.x, as that will soon be the default for Git for Windows (and is already the current setup on this developer's machine).
kewillford
approved these changes
May 11, 2019
|
Thanks @kewillford! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This backports:
t5570-git-daemon.shon macOS.