Skip to content

parallel/test-runner-plan is flaky #57928

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
joyeecheung opened this issue Apr 18, 2025 · 1 comment
Open

parallel/test-runner-plan is flaky #57928

joyeecheung opened this issue Apr 18, 2025 · 1 comment
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI. test_runner Issues and PRs related to the test runner subsystem.

Comments

@joyeecheung
Copy link
Member

joyeecheung commented Apr 18, 2025

Test

test-runner-plan

Platform

Windows x64

Reason parallel/test-runner-plan
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/57866/, https://github.com/nodejs/node/pull/57438/, https://github.com/nodejs/node/pull/55337/, https://github.com/nodejs/node/pull/57871/, https://github.com/nodejs/node/pull/57879/)
Appeared test-azure_msft-win2016_vs2017-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/66243/
Last CI https://ci.nodejs.org/job/node-test-pull-request/66270/

Console output

Example

not ok 713 parallel/test-runner-plan
  ---
  duration_ms: 2169.01200
  severity: fail
  exitcode: 1
  stack: |-
    Test failure: 'throws if options is not an object'
    Location: test\parallel\test-runner-plan.mjs:9:3
    AssertionError [ERR_ASSERTION]: Missing expected exception.
        at assert.<computed> [as throws] (node:internal/test_runner/test:254:18)
        at TestContext.<anonymous> (file:///C:/workspace/node-test-binary-windows-js-suites/node/test/parallel/test-runner-plan.mjs:10:14)
        at Test.runInAsyncScope (node:async_hooks:214:14)
        at Test.run (node:internal/test_runner/test:979:25)
        at Test.start (node:internal/test_runner/test:877:17)
        at node:internal/test_runner/test:1356:71
        at node:internal/per_context/primordials:483:82
        at new Promise (<anonymous>)
        at new SafePromise (node:internal/per_context/primordials:451:29)
        at node:internal/per_context/primordials:483:9 {
      generatedMessage: false,
      code: 'ERR_ASSERTION',
      ...

Additional information

From the reliability reports, this seems to be x64 Windows only.

First report where it failed 3 PRs out of the last 100 CI runs: https://github.com/nodejs/reliability/blob/main/reports/2025-04-05.md
Latest report (today) where it failed 5 PRs out of the last 100 CI runs: https://github.com/nodejs/reliability/blob/main/reports/2025-04-18.md

The test was introduced in #57304

@joyeecheung joyeecheung added flaky-test Issues and PRs related to the tests with unstable failures on the CI. test_runner Issues and PRs related to the test runner subsystem. labels Apr 18, 2025
@joyeecheung
Copy link
Member Author

cc @nodejs/test_runner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI. test_runner Issues and PRs related to the test runner subsystem.
Projects
None yet
Development

No branches or pull requests

1 participant