Skip to content

Test clang-tools-extra/test/clang-doc/basic-project.test failing when built using non-ninja generator #97507

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dyung opened this issue Jul 3, 2024 · 2 comments · Fixed by #97540, #98099 or #101387

Comments

@dyung
Copy link
Collaborator

dyung commented Jul 3, 2024

We have an internal job which attempts to build the LLVM tree using the Visual Studio 2019 builder configuration (non-ninja) and it recently started hitting a test failure of clang-tools-extra/test/clang-doc/basic-project.test which I bisected back to when f14ad74 (#96358) was committed. The problem is that the change hard-codes how to find the dependent assets as ../share/clang-doc. This is fine in ninja builds, but fails with the Visual Studio generator because the configuration name is part of the path.

For example, in a ninja build, the value of AssetsPath is (correctly) C:\Dev\git\merge\build\bin\..\share\clang-doc\index.js. But when using a Visual Studio generator, the value of AssetsPath is now C:\Dev\git\merge\build\Debug\bin\..\share\clang-doc\index.js which refers to a non-existent directory. In this particular case, it requires an extra .. in the path to find the expected directory.

Lit test output:

FAIL: Clang Tools :: clang-doc/basic-project.test (1 of 1)
******************** TEST 'Clang Tools :: clang-doc/basic-project.test' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 1
rm -rf C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp && mkdir -p C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/docs C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/build
# executed command: rm -rf 'C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp'
# executed command: mkdir -p 'C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/docs' 'C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/build'
# RUN: at line 2
sed 's|$test_dir|C:/Dev/git/merge/clang-tools-extra/test/clang-doc|g' C:\Dev\git\merge\clang-tools-extra\test\clang-doc/Inputs/basic-project/database_template.json > C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/build/compile_commands.json
# executed command: sed 's|$test_dir|C:/Dev/git/merge/clang-tools-extra/test/clang-doc|g' 'C:\Dev\git\merge\clang-tools-extra\test\clang-doc/Inputs/basic-project/database_template.json'
# RUN: at line 3
clang-doc --format=html --output=C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/docs --executor=all-TUs C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/build/compile_commands.json
# executed command: clang-doc --format=html '--output=C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/docs' --executor=all-TUs 'C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/build/compile_commands.json'
# .---command stdout------------
# | Emiting docs in html format.
# `-----------------------------
# .---command stderr------------
# | default index.js file missing at C:\Dev\git\merge\build\Debug\bin\..\share\clang-doc\index.js
# |
# `-----------------------------
# error: command failed with exit status: 1
@cor3ntin
Copy link
Contributor

@cor3ntin cor3ntin reopened this Jul 26, 2024
@Endilll
Copy link
Contributor

Endilll commented Jul 26, 2024

I disabled the test in 88549cf

PeterChou1 added a commit that referenced this issue Aug 8, 2024
Fixes #97507

#96809 introduce non-determinstic behaviour in clang-doc which caused
the output to be randomly ordered which lead to randomly failing test.
This patch modify clang-doc to behave deterministically again by sorting
the output by location or USR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment