Windows: swift-package-manger blocks on send to swift-build

[kcieplak]

Swift package manager can no longer communicate properly with swift build over its dispatch queue.
This only occurs on Windows.

Swift package manager is being updated to be able to use swift-build as its backend build system. When running swift package commands using the swift-build backend and using swift 6.2, a send to swift-build blocks forever.

I have traced it down to differences in dispatch.dll

This can be reproduced as follows.

Download/Install 6.2 Toolchain

mkdir example

cd example
swift package init --type tool --name swift-build-hang
swift build --vv --build-system swiftbuild

This will hang as such (most of the time on first message to swift-build, it may get one or two messages thru)

debug: PIF: Processing 1 modules:
debug: PIF: Created target 'PACKAGE-TARGET:swift-build-hang--17122C12685F4EBB-testable' of type 'objectFile' with name 'swift-build-hang' and product name 'swift-build-hang.o'
debug: PIF:   Added '["$(inherited)"]' to imparted OTHER_LDFLAGS
debug: PIF:   Added '["$(BUILT_PRODUCTS_DIR)/PackageFrameworks", "$(inherited)"]' to imparted FRAMEWORK_SEARCH_PATHS
debug: PIF:   Added source file group 'C:\Users\kcieplak\example\Sources\swift-build-hang'
debug: PIF:     Added source file 'swift_build_hang.swift'
debug: PIF:   Added dependency on product 'PACKAGE-PRODUCT:ArgumentParser'
debug: PIF: Created project 'AGGREGATE' with name 'Aggregate'
debug: PIF:   Created target 'ALL-INCLUDING-TESTS' with name 'AllIncludingTests' and 2 (unlinked) dependencies
debug: PIF:   Created target 'ALL-EXCLUDING-TESTS' with name 'AllExcludingTests' and 2 (unlinked) dependencies

Now to revert the dispatch.dll to a 6.1.X variant.

cd <ToolchainDir>

cd "c:\Users\kcieplak\AppData\Local\Programs\Swift\Toolchains\6.2.0+Asserts\usr\bin"
copy libdispatch.dll libdispatch.dll.orig
copy "c:\Users\kcieplak\AppData\Local\Programs\Swift -6.1.1\Runtimes\6.1.0\usr\bin\dispatch.dll" dispatch.dll
Overwrite dispatch.dll? (Yes/No/All): Y
        1 file(s) copied.

Run again
swift build --vv --build-system swiftbuild

Things begin to work consistently again.

debug: PIF:   Created target 'ALL-INCLUDING-TESTS' with name 'AllIncludingTests' and 2 (unlinked) dependencies
debug: PIF:   Created target 'ALL-EXCLUDING-TESTS' with name 'AllExcludingTests' and 2 (unlinked) dependencies
info: Compute target dependency graph
Building for debugging...
0%: Computing dependencies
info: Building targets in dependency order
info: Target dependency graph (6 targets)
info: Target 'AllExcludingTests' in project 'Aggregate'
0%: Provisioning 3 / 6
info: ➜ Explicit dependency on target 'swift-build-hang-product' in project 'swift-build-hang'
info: ➜ Explicit dependency on target 'swift-build-hang' in project 'swift-build-hang'
info: Target 'swift-build-hang' in project 'swift-build-hang'
info: ➜ Explicit dependency on target 'ArgumentParser-product' in project 'swift-argument-parser'
info: Target 'swift-build-hang-product' in project 'swift-build-hang'
info: ➜ Explicit dependency on target 'ArgumentParser-product' in project 'swift-argument-parser'
.....

swift-package-manager uses a send queue that uses dispatch under the covers.

SPM message send

public func loadWorkspace(containerPath: String) async throws {
        _ = try await service.send(request: SetSessionWorkspaceContainerPathRequest(sessionHandle: uid, containerPath: containerPath))
    } 

swift-build has its messaging interface implemented using dispatch.

See https://github.com/swiftlang/swift-build/blob/main/Sources/SWBBuildService/BuildService.swift

I have locally built dispatch.dll and can confirm that the issue does not happen for the release/6.1 branch.

I suspect it has to do with this change:
#842

[compnerd]

Can you share the commit that you believe to be suspect?

[kcieplak]

I am currently reverting 1072e1c locally and will see if it is isolated this change.

[kcieplak]

So it is not 1072e1c, that is not even present on the release/6.2 branch, that was my mistake, release/6.2 has not been changed for a while.

I was able to quickly iterate with a hacked up build.ps1 so I could just build dispatch.dll then replace the toolchain version
i.e.

c:\Users\kcieplak\AppData\Local\Programs\Swift\Toolchains\6.2.0+Asserts\usr\bin>copy T:\aarch64-unknown-windows-msvc\Dispatch\dispatch.dll dispatch.dll
Overwrite dispatch.dll? (Yes/No/All): Yes
        1 file(s) copied.

I verified that we are infact using the that dispatch.dll, by inspecting the blocked process in "Process Manager"

Note the blockage does not always occur on the first send, but sometimes never happens or happens after a few messages between swift-pacakge-manager and the buildservice in swift-build.

HEAD of release/6.1 : 10/10 working
HEAD of release/6.2: 8/10 Hangs

Move back commits on release/6.2

git checkout 34bc77c5688c94bf3ce5016474c0a07f5af487bf 
HEAD is now at 34bc77c Merge pull request #865 from compnerd/cross-compilation

10/10 Working

git checkout ef7ded0b34a166fffbe5a35956c357a53f781b62
HEAD is now at ef7ded0 Switch Win32 pipes to PIPE_WAIT

9/10 Hanging

Looks like ef7ded0 is the culprit.
ef7ded0
#866

release/6.2 with revert of merge #866

commit d1d14d956576570fefab0902f4a19dbd6812b786 (HEAD -> release/6.2, tag: swift-6.2-DEVELOPMENT-SNAPSHOT-
2025-06-17-a, tag: swift-6.2-DEVELOPMENT-SNAPSHOT-2025-06-14-a, tag: swift-6.2-DEVELOPMENT-SNAPSHOT-2025-06-12-a, tag: swift-6.2-DEVELOPMENT-SNAPSHOT-2025-06-03-a, tag: swift-6.2-DEVELOPMENT-SNAPSHOT-2025-05-30-a, tag: swift-6.2-DEVELOPMENT-SNAPSHOT-2025-05-15-a, tag: swift-6.2-DEVELOPMENT-SNAPSHOT-2025-05-14-a, tag: swift-6.2-DEVELOPMENT-SNAPSHOT-2025-05-07-a, tag: swift-6.2-DEVELOPMENT-SNAPSHOT-2025-04-24-a, tag: swift-6.2-DEVELOPMENT-SNAPSHOT-2025-04-23-a, tag: swift-6.2-DEVELOPMENT-SNAPSHOT-2025-04-21-a, origin/release/6.2)
|\  Merge: 34bc77c 207a816
| | Author: Saleem Abdulrasool <compnerd@compnerd.org>
| | Date:   Wed Apr 16 09:52:25 2025 -0700
| |
| |     Merge pull request #866 from compnerd/6.2-pipes
| |
| |     IO: switch Win32 pipes to `PIPE_WAIT` and use a sentinel byte
| |
| * commit 207a8162caa1a2ea0c22f0f2012affa25468fa3a
| | Author: Jeremy Day <jeremy@thebrowser.company>
| | Date:   Fri Jan 24 14:21:30 2025 -0800
| |
| |     Clamp write to bufsize-1 bytes to afford sentinel value for no-progress
| |
| * commit ef7ded0b34a166fffbe5a35956c357a53f781b62
|/  Author: Jeremy Day <jeremy@thebrowser.company>
|   Date:   Fri Jan 24 13:59:54 2025 -0800
|
|       Switch Win32 pipes to PIPE_WAIT
|

Revert the merge commit.

C:\Users\kcieplak\swift-corelibs-libdispatch>git revert d1d14d956576570fefab0902f4a19dbd6812b786 -m 1
[release/6.2 6b157ef] Revert "Merge pull request #866 from compnerd/6.2-pipes"
 3 files changed, 17 insertions(+), 52 deletions(-)

10/10 working

So based on this I this is the PR that I believe introduced the issues.
#866

I will see if I can distill down the implementation of the swift-build build service messaging interface to something that can be a test case for this usage pattern. There were tests run on the windows platform that passed https://ci-external.swift.org/job/swift-corelibs-libdispatch-PR-windows/128/console so there must be some test gaps.

[compnerd]

Great! What that means is that my concerns about the buffer sizes wasn't entirely unfounded. We should figure out what's going on, and if it's not trivial to fix, revert the changes on 6.2. CC @z2oh

[kcieplak]

It appears that the swift-build build service under a bunch of layers uses DispatchQueue for its messaging.
See: https://github.com/swiftlang/swift-build/blob/main/Sources/SWBUtil/SWBDispatch.swift

[z2oh]

Ah shoot; I had been unable to produce a hang like this when testing #854 but I agree that seems to be what is happening here. Thank you for investigating @kcieplak and my apologies for the headache. I'll try to consider an alternate solution for #820.