Introduce Dev only module for relevant extensions

[phillip-kruger]

This is on top of the work that @aloubyansky did to allow Conditional dependency based on mode (dev mode in this case)

For now I only moved Dev UI Runtime to these new modules.
With some more refactoring we can move other things like HotReload setup and Dev console (I'll try that in another PR)

This PR is fairly big, but there is a commit per extension to make it easier to review. Basically it is the following for each extension:

• Created a new module for runtime dev
• Added this new module to the bom
• Add this new module as a dependency in the deployment module
• Add the conditional dependency in the runtime module (see https://quarkus.io/version/main/guides/conditional-extension-dependencies#dev-mode-dependencies-on-regular-maven-artifacts)
• Move any dev ui runtime code to the new module

This means that all bits of dev ui is now removed from a Prod build. Previously the dev ui runtime classes was still part of the final bundle (even though they where not use internally, if users wanted they can create new instances of these classes)

[quarkus-bot]

/cc @gsmet (hibernate-orm,hibernate-search), @marko-bekhta (hibernate-search), @yrodiere (hibernate-orm,hibernate-search)

[github-actions]

🎊 PR Preview 1c093e3 has been successfully built and deployed to https://quarkus-pr-main-45053-preview.surge.sh/version/main/guides/

• Images of blog posts older than 3 months are not available.
• Newsletters older than 3 months are not available.

[aloubyansky]

The changes seem to look good but we need to check the failures

[aloubyansky]

I'll check why this failed

[aloubyansky]

We need 8583576 for this CI

[aloubyansky]

Basically, we need to merge #43618 and rebase

[phillip-kruger]

Failures are similar to what we had in #43618 so once #43618 is merged I'll rebase this and we should have a green CI

[quarkus-bot]

Status for workflow Quarkus Documentation CI

This is the status report for running Quarkus Documentation CI on commit dbf8e8b.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

Warning

There are other workflow runs running, you probably need to wait for their status before merging.

[quarkus-bot]

Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit dbf8e8b.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.

---

Flaky tests - Develocity

⚙️ JVM Tests - JDK 17

📦 extensions/infinispan-cache/deployment

✖ io.quarkus.cache.infinispan.InfinispanCacheTest.testGetAsyncWithParallelCalls - History

• expected: "thread1" but was: "thread2" - org.opentest4j.AssertionFailedError

org.opentest4j.AssertionFailedError: 

expected: "thread1"
 but was: "thread2"
	at io.quarkus.cache.infinispan.InfinispanCacheTest.testGetAsyncWithParallelCalls(InfinispanCacheTest.java:283)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at io.quarkus.test.QuarkusUnitTest.runExtensionMethod(QuarkusUnitTest.java:521)
	at io.quarkus.test.QuarkusUnitTest.interceptTestMethod(QuarkusUnitTest.java:435)

---

⚙️ JVM Tests - JDK 21

📦 extensions/smallrye-reactive-messaging-amqp/deployment

✖ io.quarkus.smallrye.reactivemessaging.amqp.AnonymousAmqpTest.test - History

• Condition with Lambda expression in io.quarkus.smallrye.reactivemessaging.amqp.AnonymousAmqpTest was not fulfilled within 1 minutes. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: Condition with Lambda expression in io.quarkus.smallrye.reactivemessaging.amqp.AnonymousAmqpTest was not fulfilled within 1 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1160)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1129)
	at io.quarkus.smallrye.reactivemessaging.amqp.AnonymousAmqpTest.test(AnonymousAmqpTest.java:30)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)

---

⚙️ JVM Integration Tests - JDK 21

📦 integration-tests/opentelemetry-grpc-only

✖ io.quarkus.it.opentelemetry.grpc.HelloGrpcClientTest.testHello - History

• java.lang.RuntimeException: Failed to start quarkus - java.lang.RuntimeException

java.lang.RuntimeException: java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.test.junit.QuarkusTestExtension.throwBootFailureException(QuarkusTestExtension.java:695)
	at io.quarkus.test.junit.QuarkusTestExtension.interceptTestClassConstructor(QuarkusTestExtension.java:793)
	at java.base/java.util.Optional.orElseGet(Optional.java:364)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)

[aloubyansky]

Alright, the CI looks good.

[phillip-kruger]

@aloubyansky should we merge ?

[aloubyansky]

Unless @gsmet and @cescoffier had further comments, i'd merge it.

[maxandersen]

This is awesome. We can finally say truly that when dev mode does not affect prod jars :)

I'm +1 but just a few thing to check/confirm (most likely in follow up prs):

1. what is the minimal Quarkus version where this runtime-dev pattern will work?
2. docs about new extension should document this pattern
3. how does it affect Quarkiverse?
4. what is our recommendation to ext writers about when they should move to it?
5. is there an issue explaining somewhere the hibernate issue that prevents the use of dev only dep for it?

[aloubyansky]

1. what is the minimal Quarkus version where this runtime-dev pattern will work?

3.18.0.CR1

2. docs about new extension should document this pattern

+1 Not sure we reference https://quarkus.io/guides/conditional-extension-dependencies#dev-mode-only-extension-dependencies from anywhere.

3. how does it affect Quarkiverse?

It's a feature any extension can use.

4. what is our recommendation to ext writers about when they should move to it?

It allows enabling certain dependencies only in dev mode. Meaning TEST and PROD builds wouldn't download those and won't include them on the classpath.

In this PR though, we are not leveraging this feature to its full potential. Specifically, we keep Dev UI related JARs as deployment dependencies in all modes and use this feature to only promote some of them to the runtime classpath in dev mode. This still avoids leaking a few Dev UI related dependencies into prod and test runtime classpath but they are still dragged along as deployment dependencies and included in mutable-jar distributions, for example. For which Keycloak raised #47210

We'll explore further reduction in dependency graphs for non-dev modes as the next step. It's doesn't involve only Dev UI, btw.

5. is there an issue explaining somewhere the hibernate issue that prevents the use of dev only dep for it?

I don't we have an GH issue for it. The issue is about dependency exclusions not being applied to conditional dependencies in our Gradle plugin implementation. It's not a dev mode specific issue, it's a general issue. Hibernate ORM extension is simply an example of when that happens, which is why it's not included in this refactoring.
I've been working on a PoC re-implementing the Gradle dependency resolver in out plugin but it's still a WIP.

[aloubyansky]

1. what is the minimal Quarkus version where this runtime-dev pattern will work?

3.18.0.CR1

Perhaps, I misunderstood the question. The runtime-dev pattern is our (internal) pattern for now.
The Agroal dependency first started using this pattern since 3.18.0.CR1 too #43618

While this pattern has its purpose and reasons, I am not sure this is something we want to promote as a recommended one. I find it to be complicated, personally. I think a cleaner split (instead of deployment to runtime promotion) would be easier to explain and understand but it has its consequences: basically, extracting Dev UI into its own extension, which is a big conceptual change and we don't want that at this point.

What I would like to explore next is actually creating a Dev UI extension for the purpose of reducing the dependency footprint in test and prod modes but leaving the necessary Dev UI SPI as a deployment dependency in all modes to still allow extensions bundle their Dev UI integration (as they do today).

[maxandersen]

While this pattern has its purpose and reasons, I am not sure this is something we want to promote as a recommended one. I find it to be complicated, personally. I think a cleaner split (instead of deployment to runtime promotion) would be easier to explain and understand but it has its consequences: basically, extracting Dev UI into its own extension, which is a big conceptual change and we don't want that at this point.

maybe I'm missing something here - but how could the split be cleaner?

What I would like to explore next is actually creating a Dev UI extension for the purpose of reducing the dependency footprint in test and prod modes but leaving the necessary Dev UI SPI as a deployment dependency in all modes to still allow extensions bundle their Dev UI integration (as they do today).

would devui still be default included without user having to add devui extension to their projects ?

[maxandersen]

Perhaps, I misunderstood the question. The runtime-dev pattern is our (internal) pattern for now.
The Agroal dependency first started using this pattern since 3.18.0.CR1 too #43618

yes so if "runtime-dev" is our internal pattern then this shouldn't be spread to quarkiverse and default templates - but i'm failing to see how this would be different with your suggested cleaner split; should we not at all allow/encourage runtime-dev for quarkiverse when using devmode only dependencies ?

[aloubyansky]

While this pattern has its purpose and reasons, I am not sure this is something we want to promote as a recommended one. I find it to be complicated, personally. I think a cleaner split (instead of deployment to runtime promotion) would be easier to explain and understand but it has its consequences: basically, extracting Dev UI into its own extension, which is a big conceptual change and we don't want that at this point.

maybe I'm missing something here - but how could the split be cleaner?

A clean split would mean removing (most of) the Dev UI classes and resources from the "primary" deployment dependencies by moving them to their own artifacts, possibly extensions and unlinking them from the "primary" extension dependencies. They would only be pulled in in dev mode.

The runtime-dev pattern does in fact move some Dev UI classes into its own artifact - that part is common, but it's still a regular deployment dependency. So it's not like it's entirely separated from the "primary" extension dependencies.

Pushing it further though would lead to creating Dev UI extensions for "primary" extensions, which is a different pattern. I think conceptually, this is what I would prefer but extension developer experience will be more complicated, unless we come up with some simple way to achieve the same effect.

What I would like to explore next is actually creating a Dev UI extension for the purpose of reducing the dependency footprint in test and prod modes but leaving the necessary Dev UI SPI as a deployment dependency in all modes to still allow extensions bundle their Dev UI integration (as they do today).

would devui still be default included without user having to add devui extension to their projects ?

Yea, that's not changing. That's the whole purpose of dev-only dependencies.

[aloubyansky]

Perhaps, I misunderstood the question. The runtime-dev pattern is our (internal) pattern for now.
The Agroal dependency first started using this pattern since 3.18.0.CR1 too #43618

yes so if "runtime-dev" is our internal pattern then this shouldn't be spread to quarkiverse and default templates - but i'm failing to see how this would be different with your suggested cleaner split; should we not at all allow/encourage runtime-dev for quarkiverse when using devmode only dependencies ?

I am not against extensions using this pattern. It works and has its benefits. IMO, it's certainly better than removing Dev UI classes and resources from build steps.
The dependency model behind it is not that straightforward though. For example, it introduces a circular dependency (on the Quarkus level) runtime-dev -> primary-runtime -> runtime-dev with the deployment module depending on all of them.

I'm just not happy with the complexity of it all and think, perhaps, we should give it some time before we start describing it as "the way" to model Dev UI dependencies.