[clang-doc] fix flaky test in clang-doc #101387
Conversation
PeterChou1
changed the title
|
This isn't fixing anything. Its re-enabling it. I'd also suggest adding people from that bug as reviewers to help you get this re-enabled. |
This was just meant as an experiment to check what was failing in the CI pipeline |
PeterChou1
force-pushed
the
clang-doc-fix-basic-project-test
branch
from
4543ee0 to
114a572
Compare
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
PeterChou1
force-pushed
the
clang-doc-fix-basic-project-test
branch
from
399c51e to
b88d3ea
Compare
PeterChou1
force-pushed
the
clang-doc-fix-basic-project-test
branch
from
1630f5a to
2539d74
Compare
| @@ -104,6 +104,9 @@ struct Reference { | |||
|
|
|||
| bool mergeable(const Reference &Other); | |||
| void merge(Reference &&I); | |||
| bool operator<(const Reference &Other) const { | |||
There was a problem hiding this comment.
How stable are usrs? Perhaps we should be sorting on another property?
There was a problem hiding this comment.
My concern here is that we probably want clang-doc to output to be ordered consistently. While sorting by USRs may make it deterministic, it’s also likely that they won’t be ordered similarly run to run. For instance, I’d expect public class fields to be sorted by name.
There was a problem hiding this comment.
the main reason I used USR was because it was unique to every declaration std sort wouldn't deal with equal elements, but it seems from the doc llvm:;sort already deterministically sort equal elements So I changed it to sort by name
PeterChou1
force-pushed
the
clang-doc-fix-basic-project-test
branch
from
3b0c23b to
3de5ffe
Compare
PeterChou1
force-pushed
the
clang-doc-fix-basic-project-test
branch
from
d4e62cb to
fc2a139
Compare
|
In the description, its probably better to use the |
PeterChou1
force-pushed
the
clang-doc-fix-basic-project-test
branch
from
1b3ec71 to
49b39e6
Compare
| bool operator<(const SymbolInfo &Other) const { | ||
| if (DefLoc && Other.DefLoc) { | ||
| return *DefLoc < *Other.DefLoc; | ||
| } | ||
| if (Loc.size() > 0 && Other.Loc.size() > 0) { | ||
| return Loc[0] < Other.Loc[0]; | ||
| } | ||
| return extractName() < Other.extractName(); | ||
| } |
There was a problem hiding this comment.
Can you add some comments about this comparison. It seems you're ordering by DefLoc, then Loc, and lastly by Name, but why that's a good/reasonable ordering isn't obvious.
There was a problem hiding this comment.
I changed the logic up a bit, now it works by sorting by declaration location, location then name, I also added some comments explaining my logic
| /// Make the output of clang-doc deterministic by sorting the children of | ||
| /// namespaces and records. | ||
| void sortUsrToInfo(llvm::StringMap<std::unique_ptr<doc::Info>> &USRToInfo) { | ||
| for (auto &I : USRToInfo) { |
There was a problem hiding this comment.
Does USRToInfo itself need to be in a sorted/stable order?
There was a problem hiding this comment.
No since USRToInfo is generated on a per file basis so there's no need to sort it
There was a problem hiding this comment.
That doesn't necessarily rule out its own iteration order affecting the output.
There was a problem hiding this comment.
How so? StringMap iteration order is not guaranteed to be deterministic and I don't think it was affecting the tests before
| if (Loc.size() > 0 && Other.Loc.size() > 0) { | ||
| return Loc[0] < Other.Loc[0]; | ||
| } | ||
| return extractName() < Other.extractName(); |
There was a problem hiding this comment.
Looking at the overall comparison, you could probably use std::tuple to make the logic simpler. The optional value may make that harder though, as I don't recall offhand how that's normally handled.
There was a problem hiding this comment.
I'm not too familiar with what you are talking about can you give an example?
There was a problem hiding this comment.
PeterChou1
force-pushed
the
clang-doc-fix-basic-project-test
branch
from
61bf9a8 to
3768c86
Compare
PeterChou1
force-pushed
the
clang-doc-fix-basic-project-test
branch
from
3768c86 to
7255f85
Compare
| // Sort by declaration location since we want the doc to be | ||
| // generated in the order of the source code. |
There was a problem hiding this comment.
Documentation often sorts methods by name, so its easy to find things. As an example, here's Rust's Vec documentation https://doc.rust-lang.org/std/vec/struct.Vec.html. Methods are sorted in the navigation window and within the body, despite not being that way in source.
Doxygen maintains source order, so its fine to do it that way for now, but we may want to consider making that a configurable option, since I think its more user friendly to do it that way (and arguably more modern).
There was a problem hiding this comment.
LGTM, but test locally w/ LLVM_ENABLE_EXPENSIVE_CHECKS=On https://llvm.org/docs/CMake.html. If that passes, then go ahead and land this, since I think the nondeterminism in output order will be accounted for.
Fixes #97507
#96809 introduce non-determinstic behaviour in clang-doc which caused the output to be randomly ordered which lead to randomly failing test. This patch modify clang-doc to behave deterministically again by sorting the output by location or USR.