feat(test reports): storage addition #1271

timhuynh94 · 2025-03-12T19:41:10Z

This PR introduces a new feature for the Vela server that adds test report storage capabilities. The setup of S3 compatible storage follows the current structure of queue and scm setup.

Feature flag for enable/disable storage.
Admins storage endpoints
- PUT /api/v1/admin/storage/bucket admin CreateBucket (require Admin token)
- GET /api/v1/admin/storage/presign admin GetPresignedURL (require Admin token)
  - Currently will give back an URL but won't work due to proxy issue with local setup (docker)
Storage endpoints
- POST /api/v1/storage/info storage StorageInfo (requires Worker Registration token)
- GET /api/v1/storage/{bucket}/objects storage ListObjects

* refactor(pipeline): use server API types for pipeline and migrate compiler types * gci * feat: add sender rule for pipelines --------- Co-authored-by: David May <[email protected]>

* chore(lint): address existing linter issues * remove dupl from exclusion list

* enhance(build): add fork field for OIDC * fix test * integration test update

* enhance(yaml): allow for users to parse pipelines using old library * testing file for internal yaml * chore(compiler): convert unmarshaled buildkite to go-yaml * remove tests used in later PRs * lintfix * fix schema * gci

…1230) Co-authored-by: David May <[email protected]>

Co-authored-by: david may <[email protected]>

* init commit * feat(repo): add pending approval timeout * fix test * remove dead code --------- Co-authored-by: David May <[email protected]>

…r have dupl anchors in map (#1232)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

timhuynh94 · 2025-03-31T14:46:03Z

This PR introduces a new feature for the Vela server that adds test report storage capabilities. The setup of S3 compatible storage follows the current structure of queue and scm setup.

Feature flag for enable/disable storage.

Admins storage endpoints

PUT /api/v1/admin/storage/bucket admin CreateBucket (require Admin token)

GET /api/v1/admin/storage/presign admin GetPresignedURL (require Admin token)

Currently will give back an URL of minio:9000/ but changing to minio:9002/ will display object content (docker)

Storage endpoints

POST /api/v1/storage/info storage StorageInfo (requires Worker Registration token)

GET /api/v1/storage/{bucket}/objects storage ListObjects

github-actions · 2025-04-09T19:33:07Z

storage/flags.go

+import (
+	"context"
+	"fmt"
+	"github.com/urfave/cli/v3"


🚫 [golangci] _{reported by reviewdog 🐶}
File is not properly formatted (gci)

github-actions · 2025-04-09T19:33:07Z

compiler/registry/github/github.go

@@ -59,6 +59,7 @@ func New(ctx context.Context, address, token string) (*Client, error) {
 	// overwrite the github client
 	c.githubClient = gitClient

+	//nolint:revive // ignore returning unexported engine


🚫 [golangci] _{reported by reviewdog 🐶}
directive //nolint:revive // ignore returning unexported engine is unused for linter "revive" (nolintlint)

github-actions · 2025-04-11T14:14:17Z

database/hook/hook.go

@@ -77,5 +77,6 @@
 		return nil, fmt.Errorf("unable to create indexes for %s table: %w", constants.TableHook, err)
 	}

+	//nolint:revive // ignore returning unexported engine


🚫 [golangci] _{reported by reviewdog 🐶}
directive //nolint:revive // ignore returning unexported engine is unused for linter "revive" (nolintlint)

github-actions · 2025-04-11T14:14:17Z

database/repo/repo.go

@@ -77,5 +77,6 @@ func New(opts ...EngineOpt) (*Engine, error) {
 		return nil, fmt.Errorf("unable to create indexes for %s table: %w", constants.TableRepo, err)
 	}

+	//nolint:revive // ignore returning unexported engine


🚫 [golangci] _{reported by reviewdog 🐶}
directive //nolint:revive // ignore returning unexported engine is unused for linter "revive" (nolintlint)

github-actions · 2025-04-14T18:31:11Z

storage/storage.go

+)
+
+// FromCLICommand helper function to setup Minio Client from the CLI arguments.
+func FromCLICommand(ctx context.Context, c *cli.Command) (Storage, error) {


🚫 [golangci] _{reported by reviewdog 🐶}
unused-parameter: parameter 'ctx' seems to be unused, consider removing or renaming it as _ (revive)

TimHuynh and others added 30 commits December 12, 2024 09:41

ground work

a2042f3

feat: add sender rule for pipelines (#1206)

8fb48dc

* refactor(pipeline): use server API types for pipeline and migrate compiler types * gci * feat: add sender rule for pipelines --------- Co-authored-by: David May <[email protected]>

chore(lint): add struct tag align rule (#1216)

a3b1bff

chore(lint): address existing linter issues (#1218)

2245376

* chore(lint): address existing linter issues * remove dupl from exclusion list

enhance(build)!: add fork field for OIDC (#1221)

5a1f403

* enhance(build): add fork field for OIDC * fix test * integration test update

enhance(compiler): cache templates per compilation (#1224)

9a5bf00

ground work

58718a1

working admin endpoints for bucket management

8e9cd8e

working admin endpoints for object operations

3239b53

all endpoints for storage

8a95775

some working test

a4f29d0

working tests

41df509

keys for test report step

86d822d

feat: deployment config for expected params and targets (#1214)

83afd15

resolve conflicts with buildkite

8b5b79f

feat: opt-in gh app integration (#1217)

9328ecd

chore(yaml): add go-yaml types package (#1225)

b16622f

* enhance(yaml): allow for users to parse pipelines using old library * testing file for internal yaml * chore(compiler): convert unmarshaled buildkite to go-yaml * remove tests used in later PRs * lintfix * fix schema * gci

resolve conflicts

3156794

rebase

57f8f9a

enhance(yaml): allow for users to set version legacy for buildkite (#…

09da5df

…1230) Co-authored-by: David May <[email protected]>

refactor(db/build): drop source index, add event index (#1228)

114ae7b

Co-authored-by: david may <[email protected]>

feat(repo)!: add pending approval timeout (#1227)

fd16b7b

* init commit * feat(repo): add pending approval timeout * fix test * remove dead code --------- Co-authored-by: David May <[email protected]>

enhance(yaml): silent fix anchor merges in YAML maps (#1231)

68547d6

fix(webhook): use correct repo variable for enqueue (#1234)

c7c1d0a

feat(pipeline)!: add warnings to pipelines that use legacy YAML lib o…

f43e2d1

…r have dupl anchors in map (#1232)

feat: status endpoints (#1235)

f1a98ac

enhance(deploy): validate on CreateDeployment (#1236)

bf2313d

enhance(metrics): include route specific queue length (#1237)

9a532cf

chore(deps): update codecov/codecov-action action to v5 (#1222)

9f13f97

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

fix(deps): update module github.com/google/go-github/v65 to v68 (#1229)

3920eab

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

TimHuynh added 6 commits March 26, 2025 12:11

meger new golangci yml

eb0ce66

remove comments

033cee7

comments

470a166

presingurl will work on 9002

bb78f51

fix test

782109d

lint fix

bd0665a

TimHuynh and others added 13 commits March 31, 2025 15:24

fix minio UI issues

3381037

changes to nginx and local dev

ef27334

Merge branch 'main' into feat_test_report_storage_addition

f7fcd81

fix linter

0860045

add status forbidden if storage is disable

b274ea0

Merge branch 'main' into feat_test_report_storage_addition

59c3a33

fix linter

6b75406

trying to make reviewdog happy

5d4954b

missing comment

7e2aa0c

make linter happy

7286b5c

Merge branch 'main' into feat_test_report_storage_addition

4abd985

Merge branch 'main' into feat_test_report_storage_addition

18d3e89

Merge branch 'main' into feat_test_report_storage_addition

22457cb

timhuynh94 mentioned this pull request Apr 9, 2025

feat(storage): storage getInfo go-vela/sdk-go#351

Draft

timhuynh94 and others added 2 commits April 9, 2025 14:21

Merge branch 'main' into feat_test_report_storage_addition

4aa63ca

fix conflicts

971163d

github-actions bot reviewed Apr 9, 2025

View reviewed changes

Merge branch 'main' into feat_test_report_storage_addition

e36e0d3

github-actions bot reviewed Apr 11, 2025

View reviewed changes

Merge branch 'main' into feat_test_report_storage_addition

91d0260

github-actions bot reviewed Apr 14, 2025

View reviewed changes

KellyMerrick marked this pull request as draft April 14, 2025 18:57

KellyMerrick changed the title ~~feat(server): test report storage addition~~ feat(test reports): storage addition Apr 14, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(test reports): storage addition #1271

feat(test reports): storage addition #1271

Uh oh!

timhuynh94 commented Mar 12, 2025 •

edited

Loading

Uh oh!

timhuynh94 commented Mar 31, 2025

Uh oh!

github-actions bot Apr 9, 2025

Uh oh!

github-actions bot Apr 9, 2025

Uh oh!

github-actions bot Apr 11, 2025

Uh oh!

github-actions bot Apr 11, 2025

Uh oh!

github-actions bot Apr 14, 2025

Uh oh!

Uh oh!

feat(test reports): storage addition #1271

Are you sure you want to change the base?

feat(test reports): storage addition #1271

Uh oh!

Conversation

timhuynh94 commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

timhuynh94 commented Mar 31, 2025

Uh oh!

github-actions bot Apr 9, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 9, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 11, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 11, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

timhuynh94 commented Mar 12, 2025 •

edited

Loading