Skip to content

Add JSON summary report output to generate_reports.py#1654

Merged
jonesbusy merged 1 commit intojenkins-infra:mainfrom
Fikri-20:feature/json-summary-report
Mar 26, 2026
Merged

Add JSON summary report output to generate_reports.py#1654
jonesbusy merged 1 commit intojenkins-infra:mainfrom
Fikri-20:feature/json-summary-report

Conversation

@Fikri-20
Copy link
Copy Markdown
Contributor

@Fikri-20 Fikri-20 commented Mar 21, 2026

Summary

Adds a summary.json file alongside the existing summary.md, providing machine-readable output for programmatic consumption (e.g., dashboards). The JSON contains the same data as the Markdown report:

  • Total migrations and success rate
  • Failures broken down by recipe
  • List of plugins with failed migrations
  • Pull request statistics (total, open, closed, merged with percentages)

Testing done

Ran update_metadata.py and generate_reports.py locally with sample metadata files.

Step 1: Update metadata

$ python scripts/update_metadata.py
INFO: Found 2 'modernization-metadata' directories.
INFO: Processing directory: .\another-plugin\modernization-metadata
INFO: Processing file: .\another-plugin\modernization-metadata\2026-03-22T10-00-00.json
INFO: Extracted PR: jenkinsci/another-plugin#15
INFO: Processing directory: .\sample-plugin\modernization-metadata
INFO: Processing file: .\sample-plugin\modernization-metadata\2026-03-20T09-15-00.json
INFO: Processing file: .\sample-plugin\modernization-metadata\2026-03-21T14-30-00.json

Step 2: Generate reports

$ python scripts/generate_reports.py
[INFO] Generated failed_migrations.csv for plugin 'sample-plugin' at: ...\sample-plugin\reports\failed_migrations.csv
[INFO] Generated recipe file for 'upgrade-to-java11' at: ...\reports\recipes\upgrade-to-java11.json
[INFO] Generated recipe file for 'upgrade-to-java17' at: ...\reports\recipes\upgrade-to-java17.json
[INFO] Summary report generated at: ...\reports\summary.md
[INFO] JSON summary report generated at: ...\reports\summary.json

Verified summary.json output:

{
  "generatedOn": "2026-03-23 17:30:23 UTC",
  "totalMigrations": 3,
  "failedMigrations": 1,
  "successRate": 66.67,
  "failuresByRecipe": [
    {"recipeId": "upgrade-to-java17", "failures": 1}
  ],
  "pluginsWithFailures": ["sample-plugin"],
  "pullRequestStats": {
    "total": 3,
    "open": 1,
    "closed": 0,
    "merged": 2,
    "openRate": 33.33,
    "closedRate": 0.0,
    "mergeRate": 66.67
  }
}

The JSON structure mirrors the existing summary.md and follows the same pattern as the per-recipe JSON files in reports/recipes/.

Submitter checklist

  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests - that demonstrates feature works or fixes the issue

…iously only generated a Markdown summary (summary.md). This adds a machine-readable summary.json in the same reports directory, containing the same data: migration stats, failures by recipe, plugins with failures, and pull request statistics. A JSON format is more suitable for programmatic consumption, such as a visualization dashboard, compared to the Markdown format.
@Fikri-20 Fikri-20 requested a review from jonesbusy as a code owner March 21, 2026 23:10
@jonesbusy
Copy link
Copy Markdown
Collaborator

@CodexRaunak Would you help me to review? Thanks!

@PratikMane0112
Copy link
Copy Markdown
Contributor

PratikMane0112 commented Mar 22, 2026

@jonesbusy @CodexRaunak

This issue I was discussing about in informal meet, i have proposed to add logic of converting summary.md to summary.json in trusted CI pipeline itself in my draft proposal but if this PR works successfully (data consistency is main concern) then lots of efforts would be reduce and data fetching for stats visualization would be smooth.

@CodexRaunak
Copy link
Copy Markdown
Contributor

LGTM, @Fikri-20 have you tested running the script locally on the metadata?

@jonesbusy
Copy link
Copy Markdown
Collaborator

Please also restore the PR template on this and future PR.

And explain the testing done (not by an AI which looks the case here)

@Fikri-20
Copy link
Copy Markdown
Contributor Author

Fikri-20 commented Mar 23, 2026

@CodexRaunak
Yes, tested locally. Created sample metadata files and ran both scripts:

  1. update_metadata.py - processes metadata directories
  2. generate_reports.py - generates summary.json and summary.md

Updated it in the "Testing done" section in the PR description.

@Fikri-20
Copy link
Copy Markdown
Contributor Author

Fikri-20 commented Mar 23, 2026

@jonesbusy
Done. Updated the PR description with the template and added local testing evidence.

Test files were created in a separate local branch (not pushed to this PR) to verify the scripts work without polluting the repo.

@jonesbusy jonesbusy merged commit 32e57de into jenkins-infra:main Mar 26, 2026
14 checks passed
@Fikri-20 Fikri-20 deleted the feature/json-summary-report branch March 26, 2026 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants