Publish a Gradle version catalog for Spring Boot's own modules

[jnizet]

Gradle now supports version catalogs, which have the advantage of generating type-safe accessors to reference the dependencies, allowing, for example

testImplementation(libs.mockk)

with support for auto-completion.

Such version catalogs can be defined in the build itself, but can also be published and then consumed by other projects. Micronaut for example does that, allowing to do in the settings files

    versionCatalogs {
        create("mn") {
            from("io.micronaut:micronaut-bom:3.3.0")
        }
    }

and then in the build files

dependencies {
    implementation(mn.picocli)
}

Overriding versions from the imported catalog is supported too.

Publishing such a version catalog along with the Spring Boot BOM would be a nice feature, which would allow migrating away from the spring dependencies plugin, now that gradle seems to support its features (with version catalogs and with the platform plugin), and benefit from the additional advantage of type-safe accessors for dependencies.

[wilkinsona]

Thanks for the suggestion, @jnizet.

The main stumbling block that I encountered when looking at version catalogs in the past was that I don't think it's possible for a catalog to import a third-party bom or another catalog. Unless things have changed since I last looked, and looking at the API I can't see that they have, we'd have to stop using the boms of third-party libraries and instead manually declare each module in that library in our own catalog. That's possible of course, but it's quite a bit of extra work. We'd probably then need to write some validation tasks to ensure that our catalog covers everything that's in each third-party bom that we were using previously.

That said, the dependency management plugin is also a non-zero amount of work. If it's going to live on, it really needs to be modernised (probably building on Gradle's constraints support) as the current codebase only uses APIs that were available in Gradle 2.x. It's testament to Gradle's backwards compatibility that it continues to work but we can't rely on that forever. I'll flag this one for discussion at a team meeting so that we can decide how best to spend our time in this area.

[jnizet]

Great, thanks @wilkinsona .

[Fleshgrinder]

I asked for BOM support but sadly the response was that this isn't going to happen. gradle/gradle#19142

[wilkinsona]

Unfortunately, a version catalog isn't a useful replacement for the dependency management plugin for the same reasons that Gradle also has its own bom support with platform and enforcedPlatform dependencies. The dependency management plugin has two key capabilities:

1. Manage the versions of transitive dependencies
2. Easy one-line version overrides

Gradle's platform support offers 1 and version catalogs offer 2. Unfortunately, easily overriding the versions of transitive dependencies is not possible even if you use a platform and a version catalog in combination. With some regret, this means that we'll have to continue to maintain and recommend the use of the dependency management plugin. Our plan is to modernise it in time for Boot's 3.0 release towards the end of this year.

We'll leave this issue open as a version catalog for Boot would still provide some benefits. For example, auto-completion for Boot's starters would be a nice developer experience improvement. It's a fairly small improvement, though, so we don't expect to tackle it any time soon.

[Fleshgrinder]

2. is provided in many places in Gradle and can be as simple as adding dependencies { implementation("group:module:version-override") }. The version catalogs play no role here, as they are just used to declare versions that can then be used in build scripts in a type-safe manner. Resolution is entirely decoupled from this. We are using just the Spring BOM without the plugin since a long time without any issues, and we are upgrading the dependencies we need to upgrade by simply declaring a dependency on our own (which is the correct way to do so in any event, because if my code depends on it than I should declare a direct dependency on it).

Maybe I'm missing something?

[wilkinsona]

For single-module libraries declaring a dependency is indeed quite concise. Things get considerably more verbose when the library has multiple modules.

As an example, if you have a dependency on spring-boot-starter-web and you want to use a specific version of Tomcat, with the dependency management plugin you can configure tomcat.version and you're done:

ext["tomcat.version"] = "9.0.56"

Without the dependency management plugin you could declare individual dependencies but you'd have to know which dependencies to declare and what to exclude to retain the starter's behaviour. It might look something like this:

implementation("org.apache.tomcat.embed:tomcat-embed-core:9.0.56") {
    exclude group: "org.apache.tomcat", module: "tomcat-annotations-api"
}
implementation("org.apache.tomcat.embed:tomcat-embed-el:9.0.56")
implementation("org.apache.tomcat.embed:tomcat-embed-websocket:9.0.56") {
    exclude group: "org.apache.tomcat", module: "tomcat-annotations-api"
}

Another option would be to define some constraints for the three Tomcat modules or a resolution strategy to set their version but neither would be as concise or as declarative as setting the version property.

which is the correct way to do so in any event, because if my code depends on it than I should declare a direct dependency on it

I don't think this is as simple as one way being correct and, therefore, others being incorrect. I think it's subjective and comes down to a team's personal preferences. We know that many Spring Boot users enjoy the concision and ease-of-use of the various starter modules and deliberately avoid direct dependencies on every module their code depends upon in favour of a single dependency that gives them everything they need.

[Fleshgrinder]

Agree, the real problem with your example is that the vendor does not provide a platform (BOM), otherwise it would boil down to a single dependency declaration again. It is very true that this situation is as it is in the majority of the JVM ecosystem. Some vendors even make it worse by publishing artifacts under the same name as others with different versioning schemes (looking at you Confluent). A good question for your example to the vendor would also be why it forces the annotations upon us, if clearly most users have no use for them (safe to say most if Spring excludes it).

I don't think this is as simple as one way being correct and, therefore, others being incorrect. I think it's subjective and comes down to a team's personal preferences. We know that many Spring Boot users enjoy the concision and ease-of-use of the various starter modules and deliberately avoid direct dependencies on every module their code depends upon in favour of a single dependency that gives them everything they need.

We also heavily work with trusting transitives and using transitives to give us what we need. I really meant this in the context of the answer where I want to control a transitive dependency because my code requires a special version of it. In that case using the dependency extension of the plugin or declaring a dependency directly boils down to being exactly the same thing. We just have two different APIs in use.

[hakonph]

What if you crated a version catalog without versions for the dependencys you manage.
And used platform to handle the versions?

Then you get type safety for the dependencies and versions managed by the platform/spring plugin?

[madorb]

@hakonph sure, i imagine that's exactly what most people are doing. it would just be rather convenient if spring itself provided the catalog, so thousands of developers didn't have to do the duplicative work of defining the catalog entries for all the various spring projects - when it could be done once upstream and imported. Not a major need, but it's a nice little "Quality of Life" improvement.

[ChristianCiach]

I second the idea of @hakonph . If Spring-Boot ever releases a version catalog, all dependencies should be defined without their versions, except the entries for the spring-boot-dependencies-BOM and the Spring-Boot-Gradle-Plugin. This way users can be forced to include the BOM as a (possibly enforced) platform dependency (or to use the spring-dependency-management plugin):

dependencies {
  implementation(enforcedPlatform(springBootLibs.bom))
  implementation(springBootLibs.starter.web)

I also think the version catalog should not duplicate the complete Spring-Boot-BOM. It should only contain the libraries of the org.springframework maven group.

That being said: Micronaut auto-generates its version catalog by converting their BOM. I would be fine with that, too.

[jamesbradlee]

This would be nice!

[austinarbor]

I also had asked the Gradle team about BOM support in gradle/gradle#26048 and they rejected my idea as well...so I am working on a settings plugin austinarbor/version-catalog-generator which automatically generates the version catalog based on a dependency in your libs.versions.toml (or another catalog file), or by specifying any other GAV coordinates you'd like. It creates a library for the direct dependencies from the specified BOM, and creates library entries from all other transitive BOM dependencies in a BFS manner. For example, the Spring BOM imports the Mockito BOM, so the dependencies in the Mockito BOM will also be included. Bundles are also automatically created based on version properties. For example, in the spring bom, there would be a bundle created with all of the dependencies that share the activemq.version property. TBD if this functionality is useful, but it exists for now.

It's still in alpha so it's a little rough around the edges and could use more customization options, but it generally seems to be working pretty well already. The API is better in the Kotlin DSL but I am trying to think of how to improve the Groovy DSL. I think the biggest missing piece is easily forcing a different version (like the ext['property']='version') functionality, but I plan on implementing something for that in the future.

Sorry for the self-promo but I stumbled upon this issue and thought the plugin I am working on could be of use here. If it's against the rules I will remove this comment. If anyone finds the plugin useful or can think of how to make it work better for the use cases described in here I will happily hear your feedback!

[wilkinsona]

Thanks for sharing, @austinarbor. Your comment is absolutely fine. Good luck with the project.

[wilkinsona]

A version catalog (or settings plugin) would also help with managing the versions of third-party plugins, keeping them up-to-date and/or aligned with other dependencies from the same project. I've opened #37836.

[oesolutions]

Playing around with some ideas.
oesolutions/spring-boot@main...oesolutions:spring-boot:catalog2
Example usage:
settings.gradle.kts

plugins {
    id("org.springframework.boot.settings") version "3.2.1-SNAPSHOT"
}
// OR if not using the above settings plugin:
dependencyResolutionManagement {
    versionCatalogs {
        val spring by registering {
            from("org.springframework.boot:spring-boot-catalog:3.2.1-SNAPSHOT")
        }
    }
}

build.gradle.kts

plugins {
    // if using the settings plugin, unforunately cannot use the plugin alias from the
    // catalog due to classloader Gradle'isms (I have a fix in mind)
    id("org.springframework.boot")
    // OR if not using the settings plugin:
    alias(spring.plugins.springBoot)
}
dependencies {
    implementation(platform(spring.springBootDependencies))
    implementation(spring.springBootStarterWeb)
}

Also experimenting with using the same configuration data in the spring-boot-dependencies build, but ran into issues that I still need to resolve.
oesolutions/spring-boot@main...oesolutions:spring-boot:catalog

Any feedback on either approach welcome.

[xenoterracide]

I'll say this, I personally don't want/need this as a replacement for a BOM, I just don't want to write the dependencies out one at a time to get the static accessors. I'd be plenty happy if spring generated a flat version of a "libs.versions.toml" file that I had to get from the repo, that had no versions defined in it.

[oesolutions]

I'd be plenty happy if spring generated a flat version of a "libs.versions.toml" file that I had to get from the repo, that had no versions defined in it.

That is what my experiments are doing. I added support to the spring boot build to publish a catalog toml file along with the other publications. In your settings.gradle you register a new catalog based on that dependency (I called it spring in my example). I forgot to show the necessary dependencyResolutionManagement.repositories block for resolving that catalog dependency.

Excerpt of the published catalog:

$ cat ~/.m2/repository/org/springframework/boot/spring-boot-catalog/3.2.1-SNAPSHOT/spring-boot-catalog-3.2.1-SNAPSHOT.toml
#
# This file has been generated by Gradle and is intended to be consumed by Gradle
#
[metadata]
format.version = "1.1"

[versions]
springBoot = "3.2.1-SNAPSHOT"

[libraries]
springBoot = {group = "org.springframework.boot", name = "spring-boot", version = "" }
springBootActuator = {group = "org.springframework.boot", name = "spring-boot-actuator", version = "" }
springBootActuatorAutoconfigure = {group = "org.springframework.boot", name = "spring-boot-actuator-autoconfigure", version = "" }
springBootAutoconfigure = {group = "org.springframework.boot", name = "spring-boot-autoconfigure", version = "" }
springBootAutoconfigureProcessor = {group = "org.springframework.boot", name = "spring-boot-autoconfigure-processor", version = "" }
springBootTestAutoconfigure = {group = "org.springframework.boot", name = "spring-boot-test-autoconfigure", version = "" }
springBootTestcontainers = {group = "org.springframework.boot", name = "spring-boot-testcontainers", version = "" }
...

[plugins]
springBoot = {id = "org.springframework.boot", version.ref = "springBoot" }

I personally don't want/need this as a replacement for a BOM

The BOM is not being replaced, my example pulls its definition from the above published catalog.

$ grep spring-boot-dependencies ~/.m2/repository/org/springframework/boot/spring-boot-catalog/3.2.1-SNAPSHOT/spring-boot-catalog-3.2.1-SNAPSHOT.toml
springBootDependencies = {group = "org.springframework.boot", name = "spring-boot-dependencies", version.ref = "springBoot" }

The idea of a org.springframework.boot.settings plugin was to abstract/automate the registration of the catalog based on the plugin's version, but it's not required.

[austinarbor]

As a user, I think a catalog without the versions diminishes the value. The original intent of version catalogs is to align versions between modules. With projects like spring and micronaut, I think it then becomes more desirable to use a version catalog in a single-module project so you can use typed dependencies without the manual effort of declaring them one-by-one yourself. However, introducing a catalog without the versions would then break the original intent of the version catalog. If I have a multi-module project, one with spring and one without, but I want to use the same dependency versions as the spring module in my non-spring module, the catalog without versions won't help me do that. I would need to add spring as a dependency to make it work...which I think defeats the original purpose?

As I mentioned above, you can use my settings plugin here to use any BOM as a version catalog (including the versions, and the ability to override any version).

I am definitely in support of a spring-native solution, but I think anything besides a fully declared catalog including all versions etc would fall short of expectations.

[oesolutions]

If I was now working on what I did back in Nov, I would agree with you and put the versions in the catalog. However, even with a no-ver-catalog, the non-spring modules could still apply the spring bom/platform to resolve the same versions (I'll note that my catalog has a version for the spring boot bom that matches the catalog version).

I think anything besides a fully declared catalog including all versions etc would fall short of expectations

I was mostly focusing on how to fit the catalog build into the spring boot build, I left it up for debate as to whether the catalog should include all spring boot deps or just the published spring boot modules. My first branch was one that tried to tie the catalog creation into the existing bom build, allowing the spring maintainers to control what goes into the catalog or not. I remember I ran into an issue causing me to create my second branch/POC, but I cannot recall what the issue was and would have to open it up again. I was hoping to get feedback from the maintainers before I invest further effort towards making a PR.

In general, I'm on the same page as you @austinarbor. Also, nice plugin!

[xenoterracide]

@wilkinsona any chance I can get the location of that code generating that part of the asciidoc that I asked for in #37836 anyways?

[wilkinsona]

It's DocumentConstrainedVersions that writes out the Asciidoc table containing all the constrained versions. It's configured in spring-boot-docs. Those constraints are extracted using ExtractConstrainedVersions that's configured by convention.

[heruan]

I read this through but I did not get what is blocking a public version catalog. I understand that it's not possible to include third-party BOMs, but it should be doable to publish a version catalog for Spring own modules.

[xenoterracide]

from what I can tell it's because @wilkinsona thinks that a version catalog only exists to replace a bill of materials; when a version catalog can provide no versions. I simply delayed in working on this; but that'll be a 3rd party project once I do.

[wilkinsona]

@wilkinsona thinks that a version catalog only exists to replace a bill of materials; when a version catalog can provide no versions

I am aware that it doesn't have to provide versions. It's hard not to be given that it has been discussed in this very issue where other members of the community have been both for and against such a version catalog.

I read this through but I did not get what is blocking a public version catalog

There's nothing specifically blocking this issue (otherwise it would be labelled as blocked) but we need to do some design work to decide exactly what we want to do (there's no clear consensus in this issue) and then prioritise doing it. We have quite a lot of other things on ours plates at the moment so it's unlikely that this will make it to the top of the priority list soon.