Skip to content

fix(github_graphql): Update Extract Jobs task to use Check Run instea… #8384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 21, 2025

Conversation

benduncan-tote
Copy link
Contributor

@benduncan-tote benduncan-tote commented Apr 15, 2025

…d of Check Suite

⚠️ Pre Checklist

Please complete ALL items in this checklist, and remove before submitting

  • I have read through the Contributing Documentation.
  • I have added relevant tests.
  • I have added relevant documentation.
  • I will add labels to the PR, such as pr-type/bug-fix, pr-type/feature-development, etc.

Summary

What does this PR do?
Refactors CheckRun into separate struct for use in Extract Jobs task.
Uses CheckRun to marshal raw collected jobs instead of CheckSuite

Does this close any open issues?

Closes #8383

Screenshots

Include any relevant screenshots here.

Other Information

Any other information that is important to this PR.

@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. component/plugins This issue or PR relates to plugins pr-type/bug-fix This PR fixes a bug labels Apr 15, 2025
@klesh
Copy link
Contributor

klesh commented Apr 17, 2025

LGTM, did you test it?

@benduncan-tote
Copy link
Contributor Author

I tested the change locally and was able to correctly get the check runs into the cicd_tasks domain table:

select * from _raw_github_graphql_jobs where id=24916;

+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|id        |24916                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|params    |{"ConnectionId":1,"Name":"*owner*/*repo*"}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|data      |{"Id":"CR_kwDOOJ953c8AAAAJFkJoag","Name":"*workflow*","DetailsUrl":"https://github.com/*owner*/*repo*/actions/runs/13944399233/job/39028156522","DatabaseId":39028156522,"Status":"COMPLETED","StartedAt":"2025-03-19T10:33:04Z","Conclusion":"SUCCESS","CompletedAt":"2025-03-19T10:33:40Z","Steps":{"TotalCount":0,"Nodes":[]}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|url       |query($node__0__id:ID!$node__1__id:ID!$node__2__id:ID!$pageSize:Int!$skipCursor:String){rateLimit{cost},node__0:node(id: $node__0__id){id,__typename,... on CheckSuite{workflowRun{databaseId},checkRuns(first: $pageSize, after: $skipCursor){totalCount,pageInfo{endCursor,hasNextPage},nodes{id,name,detailsUrl,databaseId,status,startedAt,conclusion,completedAt,steps(first: 50){totalCount,nodes{completedAt,conclusion,name,number,secondsToCompletion,startedAt,status}}}}}},node__1:node(id: $node__1__id){id,__typename,... on CheckSuite{workflowRun{databaseId},checkRuns(first: $pageSize, after: $skipCursor){totalCount,pageInfo{endCursor,hasNextPage},nodes{id,name,detailsUrl,databaseId,status,startedAt,conclusion,completedAt,steps(first: 50){totalCount,nodes{completedAt,conclusion,name,number,secondsToCompletion,startedAt,status}}}}}},node__2:node(id: $node__2__id){id,__typename,... on CheckSuite{workflowRun{databaseId},checkRuns(first: $pageSize, after: $skipCursor){totalCount,pageInfo{endCursor,hasNextPage},nodes{id,name,detailsUrl,databaseId,status,startedAt,conclusion,completedAt,steps(first: 50){totalCount,nodes{completedAt,conclusion,name,number,secondsToCompletion,startedAt,status}}}}}}}|
+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|input     |{"node": [{"id": "CS_kwDOOJ953c8AAAAIW-pyig"}, {"id": "CS_kwDOOJ953c8AAAAIW-TNvQ"}, {"id": "CS_kwDOOJ953c8AAAAIW-RZKQ"}], "pageSize": 20, "skipCursor": null}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|created_at|2025-04-15 14:02:39.041                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

select * from _tool_github_jobs where _raw_data_id=24916;

+----------------+--------------------------------------------------------------------------------------------+
|created_at      |2025-04-15 15:07:39.816                                                                     |
+----------------+--------------------------------------------------------------------------------------------+
|updated_at      |2025-04-15 15:07:39.816                                                                     |
+----------------+--------------------------------------------------------------------------------------------+
|_raw_data_params|{"ConnectionId":1,"Name":"*owner*/*repo*"}                                |
+----------------+--------------------------------------------------------------------------------------------+
|_raw_data_table |_raw_github_graphql_jobs                                                                    |
+----------------+--------------------------------------------------------------------------------------------+
|_raw_data_id    |24916                                                                                       |
+----------------+--------------------------------------------------------------------------------------------+
|_raw_data_remark|                                                                                            |
+----------------+--------------------------------------------------------------------------------------------+
|connection_id   |1                                                                                           |
+----------------+--------------------------------------------------------------------------------------------+
|repo_id         |949975517                                                                                   |
+----------------+--------------------------------------------------------------------------------------------+
|id              |39028156522                                                                                 |
+----------------+--------------------------------------------------------------------------------------------+
|run_id          |39028156522                                                                                 |
+----------------+--------------------------------------------------------------------------------------------+
|run_url         |                                                                                            |
+----------------+--------------------------------------------------------------------------------------------+
|node_id         |CR_kwDOOJ953c8AAAAJFkJoag                                                                   |
+----------------+--------------------------------------------------------------------------------------------+
|head_sha        |                                                                                            |
+----------------+--------------------------------------------------------------------------------------------+
|url             |                                                                                            |
+----------------+--------------------------------------------------------------------------------------------+
|html_url        |https://github.com/*owner*/*repo*/actions/runs/13944399233/job/39028156522|
+----------------+--------------------------------------------------------------------------------------------+
|status          |COMPLETED                                                                                   |
+----------------+--------------------------------------------------------------------------------------------+
|conclusion      |SUCCESS                                                                                     |
+----------------+--------------------------------------------------------------------------------------------+
|started_at      |2025-03-19 10:33:04.000                                                                     |
+----------------+--------------------------------------------------------------------------------------------+
|completed_at    |2025-03-19 10:33:40.000                                                                     |
+----------------+--------------------------------------------------------------------------------------------+
|name            |*workflow*                              |
+----------------+--------------------------------------------------------------------------------------------+
|steps           |[]                                                                                          |
+----------------+--------------------------------------------------------------------------------------------+
|check_run_url   |                                                                                            |
+----------------+--------------------------------------------------------------------------------------------+
|labels          |null                                                                                        |
+----------------+--------------------------------------------------------------------------------------------+
|runner_id       |0                                                                                           |
+----------------+--------------------------------------------------------------------------------------------+
|runner_name     |                                                                                            |
+----------------+--------------------------------------------------------------------------------------------+
|runner_group_id |0                                                                                           |
+----------------+--------------------------------------------------------------------------------------------+
|type            |                                                                                            |
+----------------+--------------------------------------------------------------------------------------------+
|environment     |PRODUCTION                                                                                  |
+----------------+--------------------------------------------------------------------------------------------+

select * from cicd_tasks where _raw_data_id=24916;

+-------------------+--------------------------------------------------------------+
|id                 |github:GithubJob:1:39028156522:39028156522                    |
+-------------------+--------------------------------------------------------------+
|created_at         |2025-04-15 15:18:52.603                                       |
+-------------------+--------------------------------------------------------------+
|updated_at         |2025-04-17 00:13:52.717                                       |
+-------------------+--------------------------------------------------------------+
|_raw_data_params   |{"ConnectionId":1,"Name":"*owner*/*repo*"}  |
+-------------------+--------------------------------------------------------------+
|_raw_data_table    |_raw_github_graphql_jobs                                      |
+-------------------+--------------------------------------------------------------+
|_raw_data_id       |24916                                                         |
+-------------------+--------------------------------------------------------------+
|_raw_data_remark   |                                                              |
+-------------------+--------------------------------------------------------------+
|name               |*workflow*|
+-------------------+--------------------------------------------------------------+
|pipeline_id        |github:GithubRun:1:949975517:39028156522                      |
+-------------------+--------------------------------------------------------------+
|result             |SUCCESS                                                       |
+-------------------+--------------------------------------------------------------+
|original_result    |SUCCESS                                                       |
+-------------------+--------------------------------------------------------------+
|status             |DONE                                                          |
+-------------------+--------------------------------------------------------------+
|original_status    |COMPLETED                                                     |
+-------------------+--------------------------------------------------------------+
|type               |                                                              |
+-------------------+--------------------------------------------------------------+
|created_date       |2025-03-19 10:33:04.000                                       |
+-------------------+--------------------------------------------------------------+
|queued_date        |null                                                          |
+-------------------+--------------------------------------------------------------+
|queued_duration_sec|null                                                          |
+-------------------+--------------------------------------------------------------+
|started_date       |2025-03-19 10:33:04.000                                       |
+-------------------+--------------------------------------------------------------+
|finished_date      |2025-03-19 10:33:40.000                                       |
+-------------------+--------------------------------------------------------------+
|duration_sec       |36                                                            |
+-------------------+--------------------------------------------------------------+
|environment        |PRODUCTION                                                    |
+-------------------+--------------------------------------------------------------+
|cicd_scope_id      |github:GithubRepo:1:949975517                                 |
+-------------------+--------------------------------------------------------------+

@klesh klesh merged commit b7fca14 into apache:main Apr 21, 2025
10 checks passed
@klesh
Copy link
Contributor

klesh commented Apr 21, 2025

Thank you for your contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/plugins This issue or PR relates to plugins pr-type/bug-fix This PR fixes a bug size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug][github_graphql] Extract Jobs task fails to extract any collected jobs
2 participants