Skip to content

Fix missing i686 dlls #591

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Feb 12, 2025
Merged

Fix missing i686 dlls #591

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
29dfcfb
check-for-missing-dlls: stop using `BITNESS`
dscho Feb 10, 2025
15be1e2
ci(check-for-missing-dlls): avoid using deprecated `--bitness`
dscho Feb 10, 2025
8ef99b3
ci(main): avoid using deprecated `--bitness`
dscho Feb 11, 2025
b7838c4
ci: fix invocation of `git.exe version`
dscho Feb 10, 2025
c9f04bb
make-file-list: stop being confused by `>=<ver>` in the `pactree` output
dscho Feb 10, 2025
1a72bc6
make-file-list: work around 32-bit issues more surgically
dscho Feb 10, 2025
574e8c3
make-file-list: work around a libunistring version problem of gnupg
dscho Feb 10, 2025
38774dc
git-extra(i686): work around a libunistring version problem of gnupg
dscho Feb 10, 2025
b9d2af4
git-extra: commit updates made by `sdk build`
dscho Feb 10, 2025
720c657
please create-sdk-artifact: add the gnupg/gnutls vs libunistring work…
dscho Feb 10, 2025
d37b18e
make-file-list.sh: exclude `lastlog2`
dscho Feb 11, 2025
8dbf89f
ci: trace more verbosely to facilitate debugging
dscho Feb 10, 2025
b1b6eb5
git-extra: fix build with `clangarm64`
dscho Feb 12, 2025
1654659
git-extra: make `pkgver` slightly smarter
dscho Feb 12, 2025
69e705a
git-extra: update checksums
dscho Feb 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 53 additions & 23 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ jobs:
shell: bash
run: |
./please.sh create-sdk-artifact \
--bitness=64 \
--architecture=x86_64 \
--sdk=.sdk \
--out=sdk-artifact \
build-installers &&
Expand Down Expand Up @@ -213,12 +213,28 @@ jobs:
fail-fast: false
matrix:
artifact: ['minimal', 'makepkg-git', 'build-installers', 'full']
bitness: ['32', '64']
arch:
- name: i686
mingw-prefix: mingw32
msystem: MINGW32
sdk-repo: git-sdk-32
- name: x86_64
mingw-prefix: mingw64
msystem: MINGW64
sdk-repo: git-sdk-64
exclude:
- artifact: minimal
bitness: 32
arch:
name: i686
mingw-prefix: mingw32
msystem: MINGW32
sdk-repo: git-sdk-32
- artifact: makepkg-git
bitness: 32
arch:
name: i686
mingw-prefix: mingw32
msystem: MINGW32
sdk-repo: git-sdk-32
steps:
- uses: actions/checkout@v4
- name: initialize bare SDK clone
Expand All @@ -229,54 +245,58 @@ jobs:
*) partial=--filter=blob:none;;
esac &&
git clone --bare --depth=1 --single-branch --branch=main $partial \
https://github.com/git-for-windows/git-sdk-${{ matrix.bitness }} .sdk
https://github.com/git-for-windows/${{ matrix.arch.sdk-repo }} .sdk
- name: build ${{ matrix.artifact }} artifact
id: build-artifact
shell: bash
run: |
set -x &&
case "${{ matrix.artifact }}" in
full)
git --git-dir=.sdk worktree add --detach sdk-artifact
;;
*)
./please.sh create-sdk-artifact \
--bitness=${{ matrix.bitness }} \
--architecture=${{ matrix.arch.name }} \
--sdk=.sdk \
--out=sdk-artifact \
${{ matrix.artifact }}
;;
esac &&
echo "git-version=$(sdk-artifact/cmd/git.exe version)" >>$GITHUB_OUTPUT &&
ls -la sdk-artifact/ &&
version="$(sdk-artifact/${{ matrix.arch.mingw-prefix }}/bin/git.exe version)" &&
echo "git-version=$version" >>$GITHUB_OUTPUT &&
cygpath -aw "$PWD/sdk-artifact/usr/bin/core_perl" >>$GITHUB_PATH &&
cygpath -aw "$PWD/sdk-artifact/usr/bin" >>$GITHUB_PATH &&
cygpath -aw "$PWD/sdk-artifact/mingw${{ matrix.bitness }}/bin" >>$GITHUB_PATH &&
echo "MSYSTEM=MINGW${{ matrix.bitness }}" >>$GITHUB_ENV
cygpath -aw "$PWD/sdk-artifact/${{ matrix.arch.mingw-prefix }}/bin" >>$GITHUB_PATH &&
echo "MSYSTEM=${{ matrix.arch.msystem }}" >>$GITHUB_ENV &&
cat $GITHUB_PATH
- name: build installer
if: matrix.artifact == 'build-installers'
shell: bash
run: ./installer/release.sh --include-self-check --output=$PWD/installer-${{ matrix.bitness }} 0-test
run: ./installer/release.sh --include-self-check --output=$PWD/installer-${{ matrix.arch.name }} 0-test
- uses: actions/upload-artifact@v4
if: matrix.artifact == 'build-installers'
with:
name: installer-${{ matrix.bitness }}
path: installer-${{ matrix.bitness }}
name: installer-${{ matrix.arch.name }}
path: installer-${{ matrix.arch.name }}
- name: run the installer
if: matrix.artifact == 'build-installers'
shell: pwsh
run: |
$exePath = Get-ChildItem -Path installer-${{ matrix.bitness }}/*.exe | %{$_.FullName}
$exePath = Get-ChildItem -Path installer-${{ matrix.arch.name }}/*.exe | %{$_.FullName}
$installer = Start-Process -PassThru -Wait -FilePath "$exePath" -ArgumentList "/SILENT /VERYSILENT /NORESTART /SUPPRESSMSGBOXES /ALLOWDOWNGRADE=1 /ALLOWINSTALLING32ON64=1 /LOG=installer.log"
$exitCode = $installer.ExitCode
if ($exitCode -ne 0) {
Write-Host "::error::Installer failed with exit code $exitCode!"
exit 1
}
if ("${{ matrix.bitness }}" -eq 32) {
if ("${{ matrix.arch.name }}" -eq "i686") {
"${env:ProgramFiles(x86)}\Git\usr\bin" | Out-File -Encoding ascii -Append $env:GITHUB_PATH
"${env:ProgramFiles(x86)}\Git\mingw32\bin" | Out-File -Encoding ascii -Append $env:GITHUB_PATH
"${env:ProgramFiles(x86)}\Git\${{ matrix.arch.mingw-prefix }}\bin" | Out-File -Encoding ascii -Append $env:GITHUB_PATH
} else {
"$env:ProgramFiles\Git\usr\bin" | Out-File -Encoding ascii -Append $env:GITHUB_PATH
"$env:ProgramFiles\Git\mingw${{ matrix.bitness }}\bin" | Out-File -Encoding ascii -Append $env:GITHUB_PATH
"$env:ProgramFiles\Git\${{ matrix.arch.mingw-prefix }}\bin" | Out-File -Encoding ascii -Append $env:GITHUB_PATH
}
- name: show installer log
# run this even if the installation failed (actually, _in particular_ when the installation failed)
Expand Down Expand Up @@ -306,30 +326,40 @@ jobs:
strategy:
fail-fast: false
matrix:
bitness: ['32', '64']
arch:
- name: i686
mingw-prefix: mingw32
msystem: MINGW32
sdk-repo: git-sdk-32
- name: x86_64
mingw-prefix: mingw64
msystem: MINGW64
sdk-repo: git-sdk-64
steps:
- uses: actions/checkout@v4
- name: initialize bare SDK clone
shell: bash
run: |
git clone --bare --depth=1 --single-branch --branch=main --filter=blob:none \
https://github.com/git-for-windows/git-sdk-${{ matrix.bitness }} .sdk
- name: build build-installers-${{ matrix.bitness }} artifact
https://github.com/git-for-windows/${{ matrix.arch.sdk-repo }} .sdk
- name: build build-installers-${{ matrix.arch.name }} artifact
shell: bash
run: |
set -x &&
INCLUDE_OBJDUMP=t \
./please.sh create-sdk-artifact \
--bitness=${{ matrix.bitness }} \
--architecture=${{ matrix.arch.name }} \
--sdk=.sdk \
--out=sdk-artifact \
build-installers &&
cygpath -aw "$PWD/sdk-artifact/usr/bin/core_perl" >>$GITHUB_PATH &&
cygpath -aw "$PWD/sdk-artifact/usr/bin" >>$GITHUB_PATH &&
cygpath -aw "$PWD/sdk-artifact/mingw${{ matrix.bitness }}/bin" >>$GITHUB_PATH &&
echo "MSYSTEM=MINGW${{ matrix.bitness }}" >>$GITHUB_ENV
cygpath -aw "$PWD/sdk-artifact/${{ matrix.arch.mingw-prefix }}/bin" >>$GITHUB_PATH &&
echo "MSYSTEM=${{ matrix.arch.msystem }}" >>$GITHUB_ENV &&
cat $GITHUB_PATH
- name: check for missing DLLs
shell: bash
run: ./check-for-missing-dlls.sh
run: sh -x ./check-for-missing-dlls.sh
- name: check for missing DLLs (MinGit)
shell: bash
run: MINIMAL_GIT=1 ./check-for-missing-dlls.sh
36 changes: 17 additions & 19 deletions check-for-missing-dlls.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,9 @@ ARCH="$(uname -m)" ||
die "Could not determine architecture"

case "$ARCH" in
i686) BITNESS=32;;
x86_64) BITNESS=64;;
i686) MINGW_PREFIX=mingw32;;
x86_64) MINGW_PREFIX=mingw64;;
aarch64) MINGW_PREFIX=clangarm64;;
*) die "Unhandled architecture: $ARCH";;
esac

Expand All @@ -46,38 +47,36 @@ unused_dlls_file=/tmp/unused-dlls.$$.txt
tmp_file=/tmp/tmp.$$.txt
trap "rm \"$used_dlls_file\" \"$missing_dlls_file\" \"$unused_dlls_file\" \"$tmp_file\"" EXIT

all_files="$(export ARCH BITNESS && "$thisdir"/make-file-list.sh | tr A-Z a-z | grep -v '/getprocaddr64.exe$')" &&
usr_bin_dlls="$(echo "$all_files" | grep '^usr/bin/[^/]*\.dll$')" &&
mingw_bin_dlls="$(echo "$all_files" | grep '^mingw'$BITNESS'/bin/[^/]*\.dll$')" &&
dirs="$(echo "$all_files" | sed -n 's/[^/]*\.\(dll\|exe\)$//p' | sort | uniq)" &&
ARCH=$ARCH "$thisdir"/make-file-list.sh | tr A-Z a-z | grep -v '/getprocaddr64.exe$' >"$tmp_file.all" &&
usr_bin_dlls="$(grep '^usr/bin/[^/]*\.dll$' "$tmp_file.all")" &&
mingw_bin_dlls="$(grep '^'$MINGW_PREFIX'/bin/[^/]*\.dll$' "$tmp_file.all")" &&
dirs="$(sed -n 's/[^/]*\.\(dll\|exe\)$//p' "$tmp_file.all" | sort | uniq)" &&
for dir in $dirs
do
test -z "$print_dir" ||
printf "dir: $dir\\033[K\\r" >&2

case "$dir" in
usr/*) dlls="$sys_dlls$LF$usr_bin_dlls$LF";;
mingw$BITNESS/*) dlls="$sys_dlls$LF$mingw_bin_dlls$LF";;
*) dlls="$sys_dlls$LF";;
usr/*) dlls="$usr_bin_dlls$LF";;
$MINGW_PREFIX/*) dlls="$mingw_bin_dlls$LF";;
*) dlls="";;
esac

paths=$(echo "$all_files" |
sed -ne 's,[][],\\&,g' -e "s,^$dir[^/]*\.\(dll\|exe\)$,/&,p")
out="$(/usr/bin/objdump -p $paths 2>"$tmp_file")"
paths=$(sed -ne 's,[][],\\&,g' -e "s,^$dir[^/]*\.\(dll\|exe\)$,/&,p" "$tmp_file.all")
/usr/bin/objdump -p $paths 2>"$tmp_file" >"$tmp_file.ldd"
paths="$(sed -n 's|^/usr/bin/objdump: \([^ :]*\): file format not recognized|\1|p' <"$tmp_file")"
test -z "$paths" ||
out="$out$LF$(ldd $paths)"
ldd $paths >>"$tmp_file.ldd"

echo "$out" |
tr A-Z\\r a-z\ |
tr A-Z\\r a-z\ <"$tmp_file.ldd" |
grep -e '^.dll name:' -e '^[^ ]*\.\(dll\|exe\):' -e '\.dll =>' |
while read a b c d
do
case "$a,$b" in
*.exe:,*|*.dll:,*) current="${a%:}";;
*.dll,"=>") # `ldd` output
echo "$a" >>"$used_dlls_file"
case "$dlls" in
case "$sys_dlls$LF$dlls" in
*"/$a$LF"*) ;; # okay, it's included
*)
echo "$current is missing $a" >&2
Expand All @@ -87,7 +86,7 @@ do
;;
dll,name:) # `objdump -p` output
echo "$c" >>"$used_dlls_file"
case "$dlls" in
case "$sys_dlls$LF$dlls" in
*"/$c$LF"*) ;; # okay, it's included
*)
echo "$current is missing $c" >&2
Expand All @@ -105,8 +104,7 @@ used_dlls_regex="/\\($(test -n "$MINIMAL_GIT" || printf 'p11-kit-trust\\|';
uniq |
sed -e 's/+x/\\+/g' -e 's/\.dll$/\\|/' -e '$s/\\|//' |
tr -d '\n')\\)\\.dll\$"
echo "$all_files" |
grep '\.dll$' |
grep '\.dll$' "$tmp_file.all" |
grep -v \
-e "$used_dlls_regex" \
-e '^usr/lib/perl5/' \
Expand Down
2 changes: 1 addition & 1 deletion git-extra/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ $(BUILDDIR)/%.res: $(SRCDIR)/%.rc
$(WINDRES) --input $< --output $@ --output-format coff

$(BUILDDIR)/WhoUses.exe: $(BUILDDIR)/WhoUses.o $(BUILDDIR)/SystemInfo.o
$(CXX) $(CXXFLAGS) -o $@ $(LDFLAGS) $^
$(CXX) $(CXXFLAGS) -o $@ $(LDFLAGS) $^ -lmsvcrt-os
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem right.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jeremyd2019 note that it was only necessary with clang's g++ shim, not with the gcc one.


$(BUILDDIR)/WhoUses.o: $(SRCDIR)/WhoUses.cpp $(SRCDIR)/SystemInfo.h
$(BUILDDIR)/SystemInfo.o: $(SRCDIR)/SystemInfo.cpp $(SRCDIR)/SystemInfo.h
Expand Down
6 changes: 3 additions & 3 deletions git-extra/PKGBUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ pkgbase="mingw-w64-${_realname}"
pkgname=($_realname
"${MINGW_PACKAGE_PREFIX}-${_realname}")
_ver_base=1.1
pkgver=1.1.641.031e03baf
pkgver=1.1.646.1654659da
pkgrel=1
pkgdesc="Git for Windows extra files"
arch=('any')
Expand All @@ -21,7 +21,7 @@ pkgver() {
test ! -f "$(git rev-parse --git-path shallow)" || git -c http.sslbackend fetch --unshallow
rev="$(git rev-list -1 HEAD -- .)"
test -n "$(git show . |
sed -n -e '1,/^@@/d' -e '/^[-+]pkgver=/d' -e '/^[-+]/p')" ||
sed -n -e '1,/^@@/d' -e '/^[-+]pkgver=/d' -e '/^[-+]pkgver=/d' -e "/^[-+] *'[0-9a-f]\{64\}'$/d" -e '/^[-+]/p')" ||
rev="$(git rev-list -1 $rev^ -- .)"
printf "%s.%s.%s" "${_ver_base}" "$(git rev-list --count $rev -- .)" \
"$(git rev-parse --short=9 $rev)"
Expand Down Expand Up @@ -68,7 +68,7 @@ sha256sums=('8ed76d1cb069ac8568f21c431f5e23caebea502d932ab4cdff71396f4f0d5b72'
'3cd83627f1d20e1108533419fcf33c657cbcf777c3dc39fa7f13748b7d63858a'
'd51229e5ec3653782a2c09aa5ad9af8f159aba94bc28498d7f358c33399b313d'
'4716d520e7e6e0a1281bad1ae4c21e3e6442127c3030d27681162b9c40aa6b9d'
'937e25c096e0fa91f86f689a00a5bf5442c2333fbedab30239a935e682ae2e80'
'6d2e4285f5671247598446fb1876a137741b7ec87bfe1d7edbd2dc522918df60'
'd212e1bbe75a9f81443126701324c9c44c3ed5750dd9822eba754a1799ed13b3'
'402c51eba82453a76f5110f4754bb1005df507a6e4532574c2b9627ff4e1dc81'
'8433a9e72b3bc9c3bc7903b54b868399bdb17a6c8de4af4dd5450dd42859c898'
Expand Down
7 changes: 7 additions & 0 deletions git-extra/git-extra.install
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,13 @@ test -d "$TMPDIR" || test ! -d "$TMP" || {\
# Work around /etc/xml/catalog being updated using the MINGW version of xmlcatalog
! grep -q '"[CD]:/' /etc/xml/catalog ||
sed -i -e 's|"[CD]:/[^"]*/usr/|"/usr/|g' /etc/xml/catalog

# Work around an outdated i686 gnupg/gnutls build that depends on a hence-updated libunistring
if test i686 = $arch -a ! -e /usr/bin/msys-unistring-2.dll -a -e /usr/bin/msys-unistring-5.dll &&
grep msys-unistring-2 /usr/bin/msys-gnutls-30.dll 2>&1 >/dev/null
then
cp /usr/bin/msys-unistring-5.dll /usr/bin/msys-unistring-2.dll
fi
}

post_upgrade () {
Expand Down
7 changes: 7 additions & 0 deletions git-extra/git-extra.install.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,13 @@ test -d "$TMPDIR" || test ! -d "$TMP" || {\
# Work around /etc/xml/catalog being updated using the MINGW version of xmlcatalog
! grep -q '"[CD]:/' /etc/xml/catalog ||
sed -i -e 's|"[CD]:/[^"]*/usr/|"/usr/|g' /etc/xml/catalog

# Work around an outdated i686 gnupg/gnutls build that depends on a hence-updated libunistring
if test i686 = $arch -a ! -e /usr/bin/msys-unistring-2.dll -a -e /usr/bin/msys-unistring-5.dll &&
grep msys-unistring-2 /usr/bin/msys-gnutls-30.dll 2>&1 >/dev/null
then
cp /usr/bin/msys-unistring-5.dll /usr/bin/msys-unistring-2.dll
fi
}

post_upgrade () {
Expand Down
Loading