Reapply "Ignore sort_query_fuzzer_runner (#16462)" (#16470)

[alamb]

This reverts commit a0eaf51.

Which issue does this PR close?

• related to SortQueryFuzzer found a failing case on main #16452

Rationale for this change

• We thought we fixed the test with the intermittent failures on main. so we started running the tests again Revert "Ignore sort_query_fuzzer_runner (#16462)" #16470

However, then the test failed again when we re-enabled it:
https://github.com/apache/datafusion/actions/runs/15780623790/job/44485106315

Let's revert the revert

What changes are included in this PR?

Re-disable the test on main while we sort out the root cause

Are these changes tested?

Are there any user-facing changes?

[blaginin]

😀

[alamb]

Thanks @blaginin I am merging this in to get main back clean again

[adriangb]

Oh boy. @alamb should we revert #15770 altogether then? Does this test not run until it's merged into main / why want's it flagged as part of that PR?

[blaginin]

We could also set enable_dynamic_filter_pushdown to false by default?

[adriangb]

We could also set enable_dynamic_filter_pushdown to false by default?

Does that solve the issue? It didn't seem to in the last thread we had on it. If it does yes I think that's better than reverting the PR. Maybe we should do that out of caution regardless.

[alamb]

Oh boy. @alamb should we revert #15770 altogether then? Does this test not run until it's merged into main / why want's it flagged as part of that PR?

I don't think we should revert the PR given there is a way to turn it off -- let's just keep debugging on main. I'll try and find time over the weekend / next week if it is still unsolved

[alamb]

Basically from what I can tell we don't really know what the root cause is

[ozankabak]

I think it is a good idea to revert #15770 on a branch, run tests for a few rounds on CI and see if the PR is the offending PR. If so, we can then revert it on main as we search for the root cause inside the changes #15770, which will significantly narrow down the possible causes.

[adriangb]

I'm trying to reproduce locally first by running it 100 times and recording failure rate. I just ran it and got 0 failures. Not sure if this is expected or not. I'll push a branch to CI and try to get it to fail -> commit the revert -> no failures.

[adriangb]

Second local run I got 4/100 failures. What a flake.

[alamb]

Second local run I got 4/100 failures. What a flake.

😱

[adriangb]

Results:
Total runs: 497
Failures: 1
Passes: 496
Failure rate: 0.20%

😭

[ozankabak]

Is this before the suspect PR?

[adriangb]

I think it was after and I had made some edits to try to locate the section of code producing the but. But the comment was just to make the point of how flakey this is. Let's continue discussion in #16452