[Tooling] Use universal test runner ngr everywhere

[amotl]

Introduction

Introduced at #64 (review), we added a universal test runner, currently located at testing/ngr.py, which is effectively just wrapping a few other calls, to be able to start maintaining a concise incantation syntax across different CI recipes.

About

This patch expands the capabilities of testing/ngr.py to be able to invoke test suites for all different kinds of example programs or environments within this repository, and updates all CI/GHA recipes to actually use it, instead of manually enumerating corresponding commands.

/cc @hlcianfagna, @hammerhead, @karynzv

[matriv]

Thx a lot @amotl I'm always rooting for consistency, left some comments.

[matriv]

Thx, but please wait for some other reviewer, more experienced at python.

[amotl]

Thx, but please wait for some other reviewer, more experienced at python.

Thanks for the review. GH-64 would need to go in first, anyway. This PR is stacked on top of it.

[seut]

lgtm otherwise

[seut]

to run gradle tests, one would issue ./gradlew test. Or what do I miss?

[amotl]

Thanks for spotting. I may have picked this up somewhere else.

[amotl]

Maybe test would exclusively invoke the software tests, but check invokes both the linters, and the software tests?

[amotl]

It looks like check is the right choice here?

The reason check is important is because it can aggregate one or more verification tasks together. This allows us to incorporate multiple tasks into a single step rather than having to execute them one by one.

The Gradle ecosystem contains a number of plugins, such as the checkstyle plugin, that provide additional functionality to the check task.

-- https://www.baeldung.com/gradle-test-vs-check

[seut]

Ah ok I see, normally I'd prefer to explicit name tasks one want to run, but as a generic task command this looks feasible.

[matriv]

Keep in mind that ./gradle check doesn't work correctly for CrateDB (talking about < 5.5 versions, that use gradle`:

~/crate/crate-5.4 (5.4 ✘)⚑✭ ᐅ ./gradlew clean check                                                                                                                                                                                                                 (5.4|✭1)
Starting a Gradle Daemon (subsequent builds will be faster)
....................................................................................................................................................................................................................................................................
> Task :libs:shared:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
.........................................

> Task :libs:guice:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :libs:sql-parser:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :libs:shared:forbiddenApisTest FAILED

> Task :server:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':libs:shared:forbiddenApisTest'.
> No signatures were added to task; use properties 'signatures', 'bundledSignatures', 'signaturesURLs', and/or 'signaturesFiles' to define those!

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 51s
93 actionable tasks: 75 executed, 18 up-to-date

[amotl]

Thanks. ngr will need to handle CrateDB differently then. I will consider this for another iteration. Up until now, it did not need to build CrateDB yet, only auxiliary artefacts.

[matriv]

yep, it was just FYI, if you ever stumble upon this.