Non-darwin test discovery with Swift Build#8722
Conversation
owenv
requested review from
bkhouri,
bripeticca,
cmcgee1024,
daveyc123,
dschaefer2,
jakepetroules,
plemarquand and
shawnhyam
as code owners
|
Locally, I can pass a decent chunk of TestDiscoveryTests on linux. macOS will need a bit of work to skip building the test runner executables, and I expect Windows will need a few tweaks @swift-ci test |
owenv
force-pushed
the
owenv/test-discovery
branch
from
a18182a to
10bc51d
Compare
owenv
changed the title
owenv
force-pushed
the
owenv/test-discovery
branch
from
10bc51d to
401a93f
Compare
|
@swift-ci test |
owenv
force-pushed
the
owenv/test-discovery
branch
from
401a93f to
3757526
Compare
|
@swift-ci test |
bkhouri
left a comment
bkhouri
left a comment
There was a problem hiding this comment.
Thanks for converting the TestDiscoverTests from XCTest to Swift Testing. I have a couple questions/comments, but nothing blocking.
owenv
force-pushed
the
owenv/test-discovery
branch
from
633a864 to
88ccf4a
Compare
|
@swift-ci test |
owenv
force-pushed
the
owenv/test-discovery
branch
from
88ccf4a to
23a8d43
Compare
|
@swift-ci test |
owenv
force-pushed
the
owenv/test-discovery
branch
from
23a8d43 to
61c3ec0
Compare
|
@swift-ci test |
owenv
force-pushed
the
owenv/test-discovery
branch
from
9cfbd82 to
e63dc87
Compare
|
@swift-ci test |
owenv
force-pushed
the
owenv/test-discovery
branch
from
e63dc87 to
70dcd09
Compare
bkhouri
left a comment
bkhouri
left a comment
There was a problem hiding this comment.
I only reviewed the tests changes. Although the overcharge is good, we should ensure to run the TestDicoveryTests against SwiftBuild on Windows and use the withKnownIssue Swift Testing API to essentially mark the test as "expected fail". This will give us signal when the underlying Windows issue is fix, thus prompting the removal of the withKnownIssue as opposed to having to remember to augment the test.
owenv
force-pushed
the
owenv/test-discovery
branch
3 times, most recently
from
1b4de2a to
e73ba32
Compare
|
@swift-ci test |
|
@swift-ci test Windows |
|
Windows: 'TestCommandSwiftBuildTests.testListWithSkipBuildAndNoBuildArtifacts' failed (14.613 seconds) |
owenv
force-pushed
the
owenv/test-discovery
branch
from
e73ba32 to
a180198
Compare
|
@swift-ci test |
owenv
force-pushed
the
owenv/test-discovery
branch
from
a180198 to
96ab7a9
Compare
|
@swift-ci test |
| struct TestDiscoveryTests { | ||
| static var buildSystems: [BuildSystemProvider.Kind] = [BuildSystemProvider.Kind.native, .swiftbuild] | ||
|
|
||
| @Test(arguments: buildSystems) |
There was a problem hiding this comment.
suggestion (non-blocking, if-pr-update): using arguments: SupportedBuildSystemOnAllPlatforms does essentially the same thing :)
| } | ||
|
|
||
| func testNonStandardName() async throws { | ||
| @Test(.bug("https://github.com/swiftlang/swift-build/issues/13"), arguments: [BuildSystemProvider.Kind.native]) |
There was a problem hiding this comment.
question: out of curiosity, why aren't we executing this test against the SwiftBuild build system?
There was a problem hiding this comment.
There are some design questions I need to sort out related to response file quoting on non-Darwin platforms and it's not 100% clear to me marking this as a known issue is the right thing to do yet
|
@swift-ci test windows |
|
@swift-ci smoke test macOS |
|
@swift-ci test macOS |
|
macOS checkout issue |
Still needs a fair amount of work and cleanup, I'll look at breaking off some parts that can land independently. Depends on the larger patch in swiftlang/swift-build#499
4 participants
Still needs a fair amount of work and cleanup, I'll look at breaking off some parts that can land independently. Depends on the larger patch in swiftlang/swift-build#499