Skip to content

[DO NOT MERGE] Rebase to v2.21.0-rc1 #2071

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1,268 commits into from

Conversation

dscho
Copy link
Member

@dscho dscho commented Feb 14, 2019

This PR serves mainly to run the CI build & test, in preparation for publishing Git for Windows v2.21.0-rc1.

We will not merge this into master yet, just in case that a Git v2.20.2 will be made available before Git v2.21.0 (which, according to https://tinyurl.com/gitCal is supposed to happen on February 24th, but -rc1 already slipped by a day, so we can expect v2.21.0 to slip by a day or two, too).

This obsoletes #2060.

dscho and others added 30 commits February 14, 2019 09:40
The file compat/msvc.c includes compat/mingw.c, which means that we have
to recompile compat/msvc.o if compat/mingw.c changes.

Signed-off-by: Johannes Schindelin <[email protected]>
On MSVC (VS2008) sigset_t is not defined.

Signed-off-by: Philip Oakley <[email protected]>
This constant is not defined in MSVC's headers.

In UCRT's fcntl.h, _O_RDONLY, _O_WRONLY and _O_RDWR are defined as 0, 1
and 2, respectively. Yes, that means that UCRT breaks with the tradition
that O_RDWR == O_RDONLY | O_WRONLY.

It is a perfectly legal way to define those constants, though, therefore
we need to take care of defining O_ACCMODE accordingly.

This is particularly important in order to keep our "open() can set
errno to EISDIR" emulation working: it tests that (flags & O_ACCMODE) is
not identical to O_RDONLY before going on to test specifically whether
the file for which open() reported EACCES is, in fact, a directory.

Signed-off-by: Philip Oakley <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
VS2015 complains when using a const pointer in memcpy()/free().

Signed-off-by: Jeff Hostetler <[email protected]>
VS2015's headers already declare that struct.

Signed-off-by: Jeff Hostetler <[email protected]>
It is just called differently in MSVC's headers.

Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
The ntstatus.h header is only available in MINGW.

Signed-off-by: Jeff Hostetler <[email protected]>
MSVC can be overzealous about some warnings. Disable them.

Signed-off-by: Philip Oakley <[email protected]>
This special-cases various signals that are not supported on Windows,
such as SIGPIPE. These cause the UCRT to throw asserts (at least in
debug mode).

Signed-off-by: Jeff Hostetler <[email protected]>
With this patch, Git can be built using the Microsoft toolchain, via:

	make MSVC=1 [DEBUG=1]

Third party libraries are built from source using the open source
"vcpkg" tool set. See https://github.com/Microsoft/vcpkg

On a first build, the vcpkg tools and the third party libraries are
automatically downloaded and built. DLLs for the third party libraries
are copied to the top-level (and t/helper) directory to facilitate
debugging. See compat/vcbuild/README.

A series of .bat files are invoked by the Makefile to find the location
of the installed version of Visual Studio and the associated compiler
tools (essentially replicating the environment setup performed by a
"Developer Command Prompt"). This should find the most recent VS2015 or
VS2017 installation. Output from these scripts are used by the Makefile
to define compiler and linker pathnames and -I and -L arguments.

The build produces .pdb files for both debug and release builds.

Note: This commit was squashed from an organic series of commits
developed between 2016 and 2018 in Git for Windows' `master` branch.
This combined commit eliminates the obsolete commits related to fetching
NuGet packages for third party libraries. It is difficult to use NuGet
packages for C/C++ sources because they may be built by earlier versions
of the MSVC compiler and have CRT version and linking issues.
Additionally, the C/C++ NuGet packages that were using tended to not be
updated concurrently with the sources.  And in the case of cURL and
OpenSSL, this could expose us to security issues.

Helped-by: Yue Lin Ho <[email protected]>
Helped-by: Philip Oakley <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
We ran out GUIDs. Again. But there is no need to: we can generate them
semi-randomly from the target file name of the project.

Note: the Vcproj generator is probably only interesting for historical
reasons; nevertheless, the upcoming Vcxproj generator (to support modern
Visual Studio versions) is based on the Vcproj generator and it is
better to fix this here first.

Signed-off-by: Johannes Schindelin <[email protected]>
For regular debugging, it is pretty helpful when a debug assertion in a
running application triggers a window that offers to start the debugger.

However, when running the test suite, it is not so helpful, in
particular when the debug assertions are then suppressed anyway because
we disable the invalid parameter checking (via invalidcontinue.obj, see
the comment in config.mak.uname about that object for more information).

So let's simply disable that window in Debug Mode (it is already
disabled in Release Mode).

Signed-off-by: Johannes Schindelin <[email protected]>
Visual Studio takes the first listed application/library as the default
startup project [1].

Detect the 'git' project and place it the head of the apps list, rather
than the tail.

Export the apps list before libs list for both the projects and global
structures of the .sln file.

[1] http://stackoverflow.com/questions/1238553/
vs2008-where-is-the-startup-project-setting-stored-for-a-solution
    "In the solution file, there are a list of pseudo-XML "Project"
    entries. It turns out that whatever is the first one ends up as
    the Startup Project, unless it’s overridden in the suo file. Argh.
    I just rearranged the order in the file and it’s good."

    "just moving the pseudo-xml isn't enough. You also have to move the
    group of entries in the "GlobalSection(ProjectConfigurationPlatforms)
    = postSolution" group that has the GUID of the project you moved to
    the top. So there are two places to move lines."

Signed-off-by: Philip Oakley <[email protected]>
Ignore .dll files copied into the top-level directory.
Ignore MSVC incremental compiler output files.

Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
It is not necessary, and Visual Studio 2015 no longer supports it, anyway.

Signed-off-by: Johannes Schindelin <[email protected]>
Since 4b623d8 (MSVC: link in invalidcontinue.obj for better POSIX
compatibility, 2014-03-29), invalidcontinue.obj is linked in the MSVC
build, but it was not parsed correctly by the buildsystem. Ignore it, as
it is known to Visual Studio and will be handled elsewhere.

Also only substitute filenames ending with .o when generating the
source .c filename, otherwise we would start to expect .cbj files to
generate .obj files (which are not generated by our build)...

In the future there may be source files that produce .obj files
so keep the two issues (.obj files with & without source files)
separate.

Signed-off-by: Philip Oakley <[email protected]>
Signed-off-by: Duncan Smart <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
The Generators/ directory can contain spurious files such as editors'
backup files. Even worse, there could be .swp files which are not even
valid Perl scripts.

Let's just ignore anything but .pm files in said directory.

Signed-off-by: Johannes Schindelin <[email protected]>
The error message talked about a "lib option", but it clearly referred
to a link option.

Signed-off-by: Philip Oakley <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
The engine.pl script expects file names not to contain spaces. However,
paths with spaces are quite prevalent on Windows. Use shellwords() rather
than split() to parse them correctly.

Helped-by: Junio C Hamano <[email protected]>
Signed-off-by: Philip Oakley <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
Git's build contains steps to handle internationalization. This caused
hiccups in the parser used to generate QMake/Visual Studio project files.

As those steps are irrelevant in this context, let's just ignore them.

Signed-off-by: Philip Oakley <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
Rather than swallowing the errors, it is better to have them in a file.

To make it obvious what this is about, use the file name
'msvc-build-makedryerrors.txt'.

Further, if the output is empty, simply delete that file. As we target
Git for Windows' SDK (which, unlike its predecessor msysGit, offers Perl
versions newer than 5.8), we can use the quite readable syntax `if -f -z
$ErrsFile` (available in Perl >=5.10).

Note that the file will contain the new values of the GIT_VERSION and
GITGUI_VERSION if they were generated by the make file. They are omitted
if the release is tagged and indentically defined in their respective
GIT_VERSION_GEN file DEF_VER variables.

Signed-off-by: Philip Oakley <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
Add an option for capturing the output of the make dry-run used in
determining the msvc-build structure for easy debugging.

You can use the output of `--make-out <path>` in subsequent runs via the
`--in <path>` option.

Signed-off-by: Philip Oakley <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
Upon seeing the '-lcurl' option, point to the libcurl.lib.

While there, fix the elsif indentation.

Signed-off-by: Philip Oakley <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
Git's test suite shows tons of breakages unless Git is compiled
*without* NO_ICONV. That means, in turn, that we need to generate
build definitions *with* libiconv, which in turn implies that we
have to handle the -liconv option properly.

Signed-off-by: Johannes Schindelin <[email protected]>
With the recent changes to allow building with MSVC=1, we now pass the
/OPT:REF option to the compiler. This confuses the parser that wants to
turn the output of a dry run into project definitions for QMake and Visual
Studio:

	Unhandled link option @ line 213: /OPT:REF at [...]

Let's just extend the code that passes through options that start with a
dash, so that it passes through options that start with a slash, too.

Signed-off-by: Johannes Schindelin <[email protected]>
One time too many did this developer call the `generate` script passing
a `--make-out=<PATH>` option that was happily ignored (because there
should be a space, not an equal sign, between `--make-out` and the
path).

And one time too many, this script not only ignored it but did not even
complain. Let's fix that.

Signed-off-by: Johannes Schindelin <[email protected]>
Based on the previous patch series to be able to compile Git using
Visual C++ from the command-line, this patch offers to generate project
definitions for Visual Studio, so that Git can be developed in a modern
IDE.

Based on the generator for Visual Studio versions <= 2008 (which used
.sln/.vcproj files), this patch copy-edits the generator of the .vcproj
files to a new generator that produces .vcxproj files ready for Visual
Studio 2010 and later (or MSBuild).

As the vcpkg system (which is used to build Git's dependencies) cannot
run in parallel (it does not lock, wreaking havoc with files being
accessed and written at the same time, letting the vcpkg processes
stumble over each others' toes), we make libgit the root of the project
dependency tree and initialize the vcpkg system in this project's
PreBuildEvent.

Signed-off-by: Johannes Schindelin <[email protected]>
The entire idea of generating the VS solution makes only sense if we
generate it via Continuous Integration; otherwise potential users would
still have to download the entire Git for Windows SDK.

So let's just add a target in the Makefile that can be used to generate
said solution; The generated files will then be committed so that they
can be pushed to a branch ready to check out by Visual Studio users.

To make things even more useful, we also generate and commit other files
that are required to run the test suite, such as templates and
bin-wrappers: with this, developers can run the test suite in a regular
Git Bash (that is part of a regular Git for Windows installation) after
building the solution in Visual Studio.

Signed-off-by: Johannes Schindelin <[email protected]>
dscho and others added 23 commits February 14, 2019 09:41
…t_flush

checkout.c: enable fscache for checkout again

Signed-off-by: Johannes Schindelin <[email protected]>
This was pull request git-for-windows#1645 from ZCube/master

Support windows container.

Signed-off-by: Johannes Schindelin <[email protected]>
…_index

Enable the filesystem cache (fscache) in refresh_index().
…-gfw

fscache: use FindFirstFileExW to avoid retrieving the short name
…ter-status-gfw

status: disable and free fscache at the end of the status command
…e-gfw

fscache: add GIT_TEST_FSCACHE support
…ter-add-gfw

At the end of the add command, disable and free the fscache
…ics-gfw

fscache: add fscache hit statistics
This brings substantial wins in performance because the FSCache is now
per-thread, being merged to the primary thread only at the end, so we do
not have to lock (except while merging).

Signed-off-by: Johannes Schindelin <[email protected]>
…safe-enable-gfw

fscache: make fscache_enable() thread safe
…DirectoryFile-gfw

 fscache: teach fscache to use NtQueryDirectoryFile
Signed-off-by: Johannes Schindelin <[email protected]>
Handle Ctrl+C in Git Bash nicely

Signed-off-by: Johannes Schindelin <[email protected]>
This fixes the issue identified in

	git-for-windows#1498

where Git would not fall back to reading credentials from a Win32
Console when the credentials could not be read from the terminal via the
Bash hack (that is necessary to support running in a MinTTY).

Tested in a Powershell window.

Signed-off-by: Johannes Schindelin <[email protected]>
Add a README.md for GitHub goodness.

Signed-off-by: Johannes Schindelin <[email protected]>
For many Win32 functions, there actually exist two variants: one with
the `A` suffix that takes ANSI parameters (`char *` or `const char *`)
and one with the `W` suffix that takes Unicode parameters (`wchar_t *`
or `const wchar_t *`).

Let's be precise what we want to use.

Signed-off-by: Johannes Schindelin <[email protected]>
To avoid having to play tricks as in earlier rounds, we bit the sour
apple and rebased the `builtin-stash-rebase-v3` branch thicket onto the
commit starting Git for Windows' merging-rebase.

(The merging-rebase pulls in the previous branch thicket via a "fake
merge", i.e. a merge commit that does not actually apply any changes
from the merged commit history. This has the unfortunate side effect of
confusing `merge` into thinking that any branch that was merged into an
earlier round does not need to be merged again.)

Signed-off-by: Johannes Schindelin <[email protected]>
It was a bad idea to just remove that option from Git for Windows
v2.15.0, as early users of that (still experimental) option would have
been puzzled what they are supposed to do now.

So let's reintroduce the flag, but make sure to show the user good
advice how to fix this going forward.

We'll remove this option in a more orderly fashion either in v2.16.0 or
in v2.17.0.

Signed-off-by: Johannes Schindelin <[email protected]>
When a third-party tool periodically runs `git status` in order to keep
track of the state of the working tree, it is a bad idea to lock the
index: it might interfere with interactive commands executed by the
user, e.g. when the user wants to commit files.

Git for Windows introduced the `--no-lock-index` option a long time ago
to fix that (it made it into Git for Windows v2.9.2(3)) by simply
avoiding to write that file.

The downside is that the periodic `git status` calls will be a little
bit more wasteful because they may have to refresh the index repeatedly,
only to throw away the updates when it exits. This cannot really be
helped, though, as tools wanting to get a periodic update of the status
have no way to predict when the user may want to lock the index herself.

Sadly, a competing approach was submitted (by somebody who apparently
has less work on their plate than this maintainer) that made it into
v2.15.0 but is *different*: instead of a `git status`-only option, it is
an option that comes *before* the Git command and is called differently,
too.

Let's give previous users a chance to upgrade to newer Git for Windows
versions by handling the `--no-lock-index` option, still, though with a
big fat warning.

Signed-off-by: Johannes Schindelin <[email protected]>
The option is deprecated now, and we better make sure that keeps saying
so until we finally remove it.

Suggested by Kevin Willford.

Signed-off-by: Johannes Schindelin <[email protected]>
mingw: safeguard against compiling with `-DUNICODE`
This branch allows third-party tools to call `git status
--no-lock-index` to avoid lock contention with the interactive Git usage
of the actual human user.

Signed-off-by: Johannes Schindelin <[email protected]>
…ored-directory-gracefully

Phase out `--show-ignored-directory` gracefully
@dscho dscho changed the title [DO NOT MERGE YET] Rebase to v2.21.0-rc1 [DO NOT MERGE] Rebase to v2.21.0-rc1 Feb 14, 2019
@dscho
Copy link
Member Author

dscho commented Feb 14, 2019

Note: Git v2.21.0-rc2 will probably released on February 19th, and I plan on setting up a new PR at that time, obsoleting this one. But I just realized that I could have picked a wiser branch name, such as rebase-to-v2.21.0, in preparation for -rc2 and for final so that I do not need to open new PRs all the time. Will do so for -rc2.

@dscho dscho mentioned this pull request Feb 14, 2019
@dscho dscho closed this Feb 14, 2019
@dscho dscho deleted the rebase-to-v2.21.0-rc1 branch February 14, 2019 12:37
@dscho
Copy link
Member Author

dscho commented Feb 14, 2019

I just realized that I could have picked a wiser branch name, such as rebase-to-v2.21.0, in preparation for -rc2 and for final so that I do not need to open new PRs all the time.

I decided to do that already today, no need to wait: #2072.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.