Skip to content

fix: propagate verification failures instead of silently swallowing them#1711

Merged
jonesbusy merged 3 commits intojenkins-infra:mainfrom
Fikri-20:fix/verify-plugin-silent-failures
Apr 18, 2026
Merged

fix: propagate verification failures instead of silently swallowing them#1711
jonesbusy merged 3 commits intojenkins-infra:mainfrom
Fikri-20:fix/verify-plugin-silent-failures

Conversation

@Fikri-20
Copy link
Copy Markdown
Contributor

@Fikri-20 Fikri-20 commented Apr 17, 2026

verifyPlugin() called plugin.withoutErrors() unconditionally at the end, regardless of whether the build actually passed. The caller in start() would always see a clean plugin and continue to fork, commit, and open a PR — even when the verification build failed. Removed both withoutErrors() calls and upgraded the log from INFO to WARN.

In dry-run mode, the error is still cleared after logging so the diff preview is still shown to the user (that's intentional — you're previewing, not committing).


Testing Done

I tested it locally, and here is the before and after

Before (mainPluginModernizerTest):

[INFO] Running io.jenkins.tools.pluginmodernizer.core.impl.PluginModernizerTest
[INFO] Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.573 s

After (fix/verify-plugin-silent-failuresmvn clean test -pl plugin-modernizer-core):

[INFO] Tests run: 432, Failures: 0, Errors: 0, Skipped: 1
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:58 min
[INFO] Finished at: 2026-04-17T09:07:39+02:00

Submitter checklist

  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests - that demonstrates feature works or fixes the issue

In dry-run mode, the tool was still writing modernization metadata files
to disk and attempting to push them to the metadata repository. This
conflicts with the expected non-destructive behavior of dry-run.

Changes:
- Add !config.isDryRun() check in the finally block of process()
- Skip collectModernizationMetadata() and related operations in dry-run
- Only plugin-metadata.json (needed for processing) is now created

Before fix:
- modernization-metadata.json was created in dry-run
- metadata-plugin-modernizer/* files were created

After fix:
- modernization-metadata.json is NOT created in dry-run
- No metadata repository operations in dry-run mode

Bug: jenkins-infra#2 - Dry-run still writes metadata
verifyPlugin() called plugin.withoutErrors() twice — once inside the
failure branch and once unconditionally — so build failures after recipe
application were always silently discarded. The guard in start() that was
meant to abort the run and skip the PR on broken builds never triggered.

Remove both withoutErrors() calls so errors from verify() propagate back
to start(), which already has the correct skip logic. Upgrade the log
from INFO to WARN to match the severity.
…ails

In run mode, a verification failure after applying a recipe must prevent
commits and PRs so broken code is never pushed. In dry-run mode the user
is only previewing changes so a verification failure should be logged as a
warning but must not suppress the diff output.
@jonesbusy jonesbusy merged commit 1829d61 into jenkins-infra:main Apr 18, 2026
17 checks passed
@Fikri-20 Fikri-20 deleted the fix/verify-plugin-silent-failures branch April 18, 2026 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants