Uploading and releasing a multi-project build

[aalmiray]

This is perhaps more of a question than an actual bug/feature.

What's the minimal configuration required to publish a set of projects (the set may be a subset of all projects in the build) to Sonatype Nexus and release the staging repository once?

As I understand correctly, following the naming conventions in tasks, the plugin may create a staging repository per project. If that's the case, then multiple staging repositories would have to be closed & released, isn't it?

Thus, if it's possible to release a subset to the same staging repository so that a single staging repository is closed & released, I'd like to know how it may be done :-)

If it's not possible to have this scenario with the current release, then please consider adding it. As a reference the Bintray plugin registers a task at the root project that depends on individual bintrayUpload tasks (one per subproject). This "aggregating" task would then wait til all subtasks finish then proceed to sync uploaded artifacts to Maven Central. If not done already then this plugin would require a similar aggregating capability to close & release a single staging repository.

[szpak]

First of all, publishing (uploading) and closing/releasing of a multi-project build is supported out of the box, assuming that you want to do it for all subprojects (e.g. different module of one project, not a mono repo with separate components). See our e2e test project.

Nevertheless, close and release tasks are only orchestrated to mustRunAfter the publishing tasks. Therefore, in a one call you should be able to call only those publishing tasks that you want to (e.g. 2 of 5 suprojects) followed with the close/release tasks (just one staging repository should be explicitly initialized just before the first publishing task is called and reused by the others). Would it be enough in your case?

Please note, however, up until #19 is implemented you need to do it (publishing and closing) in a one Gradle call.

[aalmiray]

Well, that's a lot of ... Kotlin code. If I understood that example correctly, the nexus plugin is only applied to the root while the subprojects configure themselves with java-library, maven-publish, and sign plugins. If this is correct them all projects will be published to a staging repository from which closing & release may occur.

How then must a subproject be marked to be skipped from uploading to a staging repository?

[vlsi]

How then must a subproject be marked to be skipped from uploading to a staging repository?

Well, Gradle does whatever you ask it to do. If you ask it to publish all the artifacts, it does so. If you ask it to execute a subset of tasks it agrees, and it does not insist on doing all the tasks for all the projects every time 😉
That is the default behavior and it is not related to publish-plugin.

Could you clarify what do you mean by "marked to be skipped"?

What the plugin does is as follows: it adds a task to the root project to initialized the staging repository, then it rewrites all the URLs for all projects to point to the newly created staging repository. The singing and publishing are out of scope for gradle-nexus/publish-plugin.

[szpak]

Well, that's a lot of ... Kotlin code.

Some people prefers that over Maven's XML ;-). We have also a complementary variant in Groovy, but just for a single module.

As @vlsi already suggested, try to just call :project1:publishToSonatype :project2:publishToSonatype closeStagingRepository.

[aalmiray]

Good grief, what if I have more than 50+ projects (as Ikonli does). Do you expect people to:

a) manually invoke :X:publishToSonatype where x = the number of projects?
b) create a task at the root level that collects al projects that should be published to Sonatype?

Again, the reason why I ask this is that the Bintray plugin handled aggregation automatically. I get it that this plugin provides the minimum building blocks to get the job done, I'm asking for convenient options that can be turned on/off when necessary. What's the point of having convention over configuration if not to avoid defining stuff over and over again?

[aalmiray]

How then must a subproject be marked to be skipped from uploading to a staging repository?

Well, Gradle does whatever you ask it to do. If you ask it to publish all the artifacts, it does so. If you ask it to execute a subset of tasks it agrees, and it does not insist on doing all the tasks for all the projects every time 😉
That is the default behavior and it is not related to publish-plugin.

I'm well aware of that.

Could you clarify what do you mean by "marked to be skipped"?

gradle publish will invoke the publish task on all matching projects. This plugin adds publishToSonatype to all matching plugins. What I want is to avoid adding publishToSonatype (and any other tasks brought by this plugin) to a subproject (or at least make then disable by default without me explicitly saying so on each individual task (which I know can be done)).

What the plugin does is as follows: it adds a task to the root project to initialized the staging repository, then it rewrites all the URLs for all projects to point to the newly created staging repository. The singing and publishing are out of scope for gradle-nexus/publish-plugin.

I'm well aware of that.

[szpak]

@aalmiray I'm afraid, I do not fully follow your case.

Do you have a multiproject build with
a) let's say 9 publishable subprojects, but you want to be able to publish 4 of them (a fixed set) some days and 5 others (the other fixed set) some other days?
b) 9 subprojects at in general, where 3 of them are publishable (and you just want to release them to Maven Central) and 6 others are just test modules or some other auxiliary stuff that cannot be published at all?
c) anything else :-)?

[aalmiray]

That would be case b). Not all projects are publishable (for whatever reason) hence why I used the word subset in the beginning.

[aalmiray]

Add up that some subprojects are publishable, just not to Maven Central. Could be to other Maven compatible repositories (non Nexus managed), could be a Gradle plugin (which would meet the criteria for publication but to a totally different repository).

So in essence, I'd like to target just a subset of publishable subprojects with little to no configuration (conventions, let's use them) and without being forced to add custom code in build scripts (or pre compiled plugin scripts) or specifying publish targets in the command line.

[szpak]

That would be case b). Not all projects are publishable (for whatever reason) hence why I used the word subset in the beginning.

In that case, I assume there is no maven-publish plugin applied in those other projects, right? Then you can just call:

./gradlew publishToSonatype closeStagingRepository

and the plugin should automatically detect "publishable" subprojects, initialize the staging repository before the first artifacts upload and close/release that staging repository after the last artifacts are uploaded.

[szpak]

Add up that some subprojects are publishable, just not to Maven Central. Could be to other Maven compatible repositories (non Nexus managed), could be a Gradle plugin (which would meet the criteria for publication but to a totally different repository).

It's somehow more complicated. We don't have that mechanism out of the box. Probably, it will be implemented once better support for monorepos is requeted (and provided). At the time being, you could try to use your own -PreleaseToCentral flag and ignore/skip publish tasks in the subprojects that should not be published to Maven Central (in those tasks configuration).

Update. If you have a good idea how we could distinguish the projects you would like to skip (without their enumeration in the project configuration or some marker property set) please let us know.

[aalmiray]

It's somehow more complicated. We don't have that mechanism out of the box. Probably, it will be implemented once better support for monorepos is requeted (and provided). At the time being, you could try to use your own -PreleaseToCentral flag and ignore/skip publish tasks in the subprojects that should not be published to Maven Central (in those tasks configuration).

I don't see what monorepo support has to do with this question. This is for a typical multi-project build where all subprojects reside in the same repository.

Update. If you have a good idea how we could distinguish the projects you would like to skip (without their enumeration in the project configuration or some marker property set) please let us know.

Given that the plugin already configures all projects in the build, using the extension it provides could be the way to go. Add a boolean property to mark the project for inclusion (default = true).

[szpak]

I don't see what monorepo support has to do with this question.

In monorepos there is (might be) a need to release - on demand - only 3 projects (for example sharing some "subroot" or not) from 100, available in the whole repo, publishable projects. So, some kind of the mechanism we are talking about would problably be required.

Given that the plugin already configures all projects in the build, using the extension it provides could be the way to go. Add a boolean property to mark the project for inclusion (default = true).

We need to collect publishable projects in one place and synchronize the whole operation (init repo before the first upload and close/release after last). As a result this plugin is applied only in the root project and it tries to find all publishable (sub)projects. Therefore, there is no extension created in subprojects.

We could try to check for some value in the ExtraPropertyExtension (not very elegant, but probably easy to implement) or provide a way to allow people define their own logic to find desired projects.

[joebowbeer]

I'm realizing that nexusPublishing needs to be defined at the root, and therefore group and version must also be defined at the root. This is causing a lot of disruption in my Android project, where only the library subproject is published.

[jvmlet]

@aalmiray , I think you can exclude sub-module from being published to nexus by NOT applying maven-publish plugin, this also can be done dynamically in gradle script :

->sub-project-a
    buld.gradle :
    apply plugin: 'maven-publish'
->sub-project-b
    buld.gradle :
      if (someCondition){apply plugin: 'maven-publish'}
->sub-project-c
    buld.gradle:
     // 'maven-publish' plugin is NOT applied
buidl.gradle:
plugins {
    id("io.github.gradle-nexus.publish-plugin") version "«version»"
}

Then invoking ./gradlew publishToSonatype closeStagingRepository will publish sub-project-a and sub-project-b only if someCondition evaluates to true

@szpak , can you please confirm this ?