add CI and other components for tests#37
Closed
Ferret-san wants to merge 1 commit into
Closed
Conversation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.OpenSSF ScorecardScorecard details
Scanned Files
|
Contributor
|
This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+18
to
+36
| name: Lint | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: ${{ env.GO_VERSION }} | ||
| cache: true | ||
|
|
||
| - name: Run golangci-lint | ||
| uses: golangci/golangci-lint-action@v3 | ||
| with: | ||
| version: ${{ env.GOLANGCI_LINT_VERSION }} | ||
| args: --timeout=5m | ||
|
|
||
| test: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 months ago
To resolve the issue, an explicit permissions block should be added to the workflow, specifying the minimum required permissions for the jobs. Since none of the jobs seem to modify repository contents or create issues/pull requests (based on the provided code), starting with:
permissions:
contents: readat the root level (top of the workflow) is advised. This will restrict the GITHUB_TOKEN to read-only access to repository contents for all jobs unless a job overrides it with its own permissions block.
- Add the following block at the top of .github/workflows/ci.yml, after the workflow name definition (after line 1):
permissions: contents: read
- If in future a job needs increased permissions, you can override at the job level as needed.
No additional imports or dependencies are required.
Suggested changeset
1
.github/workflows/ci.yml
| @@ -1,4 +1,6 @@ | ||
| name: CI | ||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| push: |
Copilot is powered by AI and may make mistakes. Always verify output.
| cache: true | ||
|
|
||
| - name: Run golangci-lint | ||
| uses: golangci/golangci-lint-action@v3 |
Check warning
Code scanning / CodeQL
Unpinned tag for a non-immutable Action in workflow Medium
Comment on lines
+37
to
+77
| name: Test | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| matrix: | ||
| go-version: ['1.20', '1.21', '1.22'] | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: ${{ matrix.go-version }} | ||
| cache: true | ||
|
|
||
| - name: Install dependencies | ||
| run: go mod download | ||
|
|
||
| - name: Run tests with coverage | ||
| run: | | ||
| go test -v -race -coverprofile=coverage.txt -covermode=atomic ./... | ||
| go test -v -race ./... -json > test-results.json | ||
|
|
||
| - name: Upload coverage to Codecov | ||
| if: matrix.go-version == '1.21' | ||
| uses: codecov/codecov-action@v4 | ||
| with: | ||
| token: ${{ secrets.CODECOV_TOKEN }} | ||
| file: ./coverage.txt | ||
| flags: unittests | ||
| name: codecov-umbrella | ||
| fail_ci_if_error: false | ||
|
|
||
| - name: Upload test results | ||
| if: always() | ||
| uses: actions/upload-artifact@v3 | ||
| with: | ||
| name: test-results-${{ matrix.go-version }} | ||
| path: test-results.json | ||
|
|
||
| build: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 months ago
To fix the problem, we should add a permissions block with the minimum permissions needed to the workflow. Since none of the jobs appear to require write access (e.g., they don't push to the repo, comment on issues/PRs, or update release assets), the minimal recommended permissions would be contents: read at the workflow (root) level: this ensures all jobs use the most restrictive set unless they need something broader. The best way is to insert the explicit block just after name: CI, before any job or workflow triggers. No imports or extra steps are needed. Only a small change at the top of .github/workflows/ci.yml is required.
Suggested changeset
1
.github/workflows/ci.yml
| @@ -1,4 +1,6 @@ | ||
| name: CI | ||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| push: |
Copilot is powered by AI and may make mistakes. Always verify output.
|
|
||
| - name: Upload coverage to Codecov | ||
| if: matrix.go-version == '1.21' | ||
| uses: codecov/codecov-action@v4 |
Check warning
Code scanning / CodeQL
Unpinned tag for a non-immutable Action in workflow Medium
Comment on lines
+78
to
+108
| name: Build | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| matrix: | ||
| os: [linux] | ||
| arch: [amd64, arm64] | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: ${{ env.GO_VERSION }} | ||
| cache: true | ||
|
|
||
| - name: Build binary | ||
| env: | ||
| GOOS: ${{ matrix.os }} | ||
| GOARCH: ${{ matrix.arch }} | ||
| run: | | ||
| go build -v -o da-server-${{ matrix.os }}-${{ matrix.arch }} ./cmd/da-server | ||
| chmod +x da-server-${{ matrix.os }}-${{ matrix.arch }} | ||
|
|
||
| - name: Upload artifacts | ||
| uses: actions/upload-artifact@v3 | ||
| with: | ||
| name: da-server-${{ matrix.os }}-${{ matrix.arch }} | ||
| path: da-server-${{ matrix.os }}-${{ matrix.arch }} | ||
|
|
||
| integration-test: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
| path: benchmark-results.txt | ||
|
|
||
| - name: Store benchmark results | ||
| uses: benchmark-action/github-action-benchmark@v1 |
Check warning
Code scanning / CodeQL
Unpinned tag for a non-immutable Action in workflow Medium
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Delete PR images older than 30 days | ||
| uses: snok/container-retention-policy@v2 |
Check warning
Code scanning / CodeQL
Unpinned tag for a non-immutable Action in workflow Medium
Comment on lines
+12
to
+84
| name: Integration Test with Localestia | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: '1.21' | ||
| cache: true | ||
|
|
||
| - name: Start Localestia | ||
| run: | | ||
| # Pull and run localestia container | ||
| docker run -d \ | ||
| --name localestia \ | ||
| -p 26658:26658 \ | ||
| -p 9090:9090 \ | ||
| ghcr.io/celestiaorg/localestia:latest | ||
|
|
||
| # Wait for localestia to be ready | ||
| echo "Waiting for Localestia to start..." | ||
| for i in {1..60}; do | ||
| if curl -s http://localhost:26658/status > /dev/null 2>&1; then | ||
| echo "Localestia is ready!" | ||
| break | ||
| fi | ||
| echo "Waiting... ($i/60)" | ||
| sleep 2 | ||
| done | ||
|
|
||
| # Get auth token from container | ||
| export AUTH_TOKEN=$(docker exec localestia cat /home/celestia/.celestia-light/auth_token) | ||
| echo "AUTH_TOKEN=${AUTH_TOKEN}" >> $GITHUB_ENV | ||
|
|
||
| - name: Build DA Server | ||
| run: | | ||
| go build -v -o da-server ./cmd/da-server | ||
|
|
||
| - name: Run DA Server Tests | ||
| env: | ||
| CELESTIA_SERVER: http://localhost:26658 | ||
| CELESTIA_AUTH_TOKEN: ${{ env.AUTH_TOKEN }} | ||
| CELESTIA_NAMESPACE: "0000000000000000000000000000000000000000000000000000000000000000" | ||
| run: | | ||
| # Start DA server in background | ||
| ./da-server \ | ||
| --celestia.server=$CELESTIA_SERVER \ | ||
| --celestia.auth-token=$CELESTIA_AUTH_TOKEN \ | ||
| --celestia.namespace=$CELESTIA_NAMESPACE \ | ||
| --addr=127.0.0.1 \ | ||
| --port=3100 & | ||
|
|
||
| DA_SERVER_PID=$! | ||
| echo "DA Server PID: $DA_SERVER_PID" | ||
|
|
||
| # Wait for DA server to be ready | ||
| sleep 5 | ||
|
|
||
| # Run integration tests | ||
| go test -v -tags=integration ./test/integration/... | ||
|
|
||
| # Cleanup | ||
| kill $DA_SERVER_PID || true | ||
|
|
||
| - name: Cleanup | ||
| if: always() | ||
| run: | | ||
| docker stop localestia || true | ||
| docker rm localestia || true | ||
|
|
||
| integration-kurtosis: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 months ago
To fix the problem, add a permissions block to the root of the workflow YAML file (.github/workflows/integration.yml). This will explicitly define the permissions the GITHUB_TOKEN has when workflows and jobs run. The minimal safe starting point is contents: read. If any workflow step requires additional permissions (such as creating or updating pull requests, issues, etc.), those privileges should be explicitly added for those jobs only where needed.
For now, the fix is to add the following below the name: line, before on:, at the root of the YAML file:
permissions:
contents: readThis applies the restriction to all jobs in the workflow unless overridden.
Suggested changeset
1
.github/workflows/integration.yml
| @@ -1,4 +1,6 @@ | ||
| name: Integration Tests | ||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| push: |
Copilot is powered by AI and may make mistakes. Always verify output.
Comment on lines
+85
to
+144
| name: Integration Test with Kurtosis | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: '1.21' | ||
| cache: true | ||
|
|
||
| - name: Install Kurtosis | ||
| run: | | ||
| echo "Installing Kurtosis CLI..." | ||
| curl -s https://raw.githubusercontent.com/kurtosis-tech/kurtosis/main/scripts/install.sh | bash | ||
|
|
||
| - name: Start Kurtosis Engine | ||
| run: | | ||
| kurtosis engine start | ||
|
|
||
| - name: Run Kurtosis Devnet | ||
| run: | | ||
| # Create kurtosis configuration | ||
| cat > devnet-config.yaml << EOF | ||
| participants: | ||
| - el_type: op-geth | ||
| cl_type: op-node | ||
| count: 1 | ||
|
|
||
| da_params: | ||
| image: ghcr.io/celestiaorg/localestia-da-server:latest | ||
|
|
||
| altda_deploy_config: | ||
| use_altda: true | ||
| da_commitment_type: GenericCommitment | ||
| da_challenge_window: 100 | ||
| da_resolve_window: 100 | ||
| da_bond_size: 0 | ||
| da_resolver_refund_percentage: 0 | ||
| EOF | ||
|
|
||
| # Run the devnet | ||
| kurtosis run github.com/ethpandaops/optimism-package --args-file devnet-config.yaml | ||
|
|
||
| - name: Run Integration Tests | ||
| run: | | ||
| # Get service details from Kurtosis | ||
| DA_SERVER_URL=$(kurtosis port print optimism-devnet da-server http) | ||
|
|
||
| # Run tests against the devnet | ||
| go test -v -tags=kurtosis ./test/kurtosis/... | ||
|
|
||
| - name: Cleanup | ||
| if: always() | ||
| run: | | ||
| kurtosis clean -a || true | ||
| kurtosis engine stop || true | ||
|
|
||
| e2e-test: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 months ago
To fix the problem, add a permissions: key either at the top level of the workflow (applying to all jobs unless overridden by a job-specific permissions block), or directly to each job definition if fine-grained control is needed. Since none of the shown jobs require write access to repository contents, the best starting permissions are contents: read, which allows the checkout action and read-only access to repository files. For jobs that use actions like actions/upload-artifact, this does not require extra token permissions. Therefore, insert the following at the top of the workflow (after the name: key):
permissions:
contents: readNo other imports, code changes, or dependency adjustments are required.
Suggested changeset
1
.github/workflows/integration.yml
| @@ -1,4 +1,6 @@ | ||
| name: Integration Tests | ||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| push: |
Copilot is powered by AI and may make mistakes. Always verify output.
Comment on lines
+145
to
+221
| name: End-to-End Test | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: '1.21' | ||
| cache: true | ||
|
|
||
| - name: Setup Docker Compose | ||
| run: | | ||
| # Create docker-compose for e2e testing | ||
| cat > docker-compose.e2e.yml << 'EOF' | ||
| version: '3.8' | ||
|
|
||
| services: | ||
| localestia: | ||
| image: ghcr.io/celestiaorg/localestia:latest | ||
| ports: | ||
| - "26658:26658" | ||
| - "9090:9090" | ||
| healthcheck: | ||
| test: ["CMD", "curl", "-f", "http://localhost:26658/status"] | ||
| interval: 10s | ||
| timeout: 5s | ||
| retries: 5 | ||
|
|
||
| da-server: | ||
| build: | ||
| context: . | ||
| dockerfile: Dockerfile | ||
| depends_on: | ||
| localestia: | ||
| condition: service_healthy | ||
| environment: | ||
| - OP_ALTDA_CELESTIA_SERVER=http://localestia:26658 | ||
| - OP_ALTDA_CELESTIA_NAMESPACE=0000000000000000000000000000000000000000000000000000000000000000 | ||
| - OP_ALTDA_ADDR=0.0.0.0 | ||
| - OP_ALTDA_PORT=3100 | ||
| ports: | ||
| - "3100:3100" | ||
| healthcheck: | ||
| test: ["CMD", "curl", "-f", "http://localhost:3100/health"] | ||
| interval: 10s | ||
| timeout: 5s | ||
| retries: 5 | ||
| EOF | ||
|
|
||
| - name: Run E2E Tests | ||
| run: | | ||
| docker-compose -f docker-compose.e2e.yml up -d | ||
|
|
||
| # Wait for services to be ready | ||
| sleep 30 | ||
|
|
||
| # Run e2e tests | ||
| go test -v -tags=e2e ./test/e2e/... | ||
|
|
||
| - name: Collect Logs | ||
| if: failure() | ||
| run: | | ||
| docker-compose -f docker-compose.e2e.yml logs > e2e-logs.txt | ||
|
|
||
| - name: Upload Logs | ||
| if: failure() | ||
| uses: actions/upload-artifact@v3 | ||
| with: | ||
| name: e2e-logs | ||
| path: e2e-logs.txt | ||
|
|
||
| - name: Cleanup | ||
| if: always() | ||
| run: | | ||
| docker-compose -f docker-compose.e2e.yml down -v |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 months ago
The fix is to add the permissions key at the top level of the workflow to enforce the principle of least privilege for the GITHUB_TOKEN. This change restricts token access to repository contents with read-only rights, which is the minimal permission level required for most test/build workflows that do not create releases, modify code, or interact with issues or PRs via the API.
To do this, edit .github/workflows/integration.yml and insert the following at the root (anywhere before the jobs block is conventionally preferred, typically after the on: trigger but before jobs:):
permissions:
contents: readNo other changes, imports, or dependencies are needed, and no existing workflow logic will be affected.
Suggested changeset
1
.github/workflows/integration.yml
| @@ -7,6 +7,9 @@ | ||
| branches: [ main, develop ] | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| integration-localestia: | ||
| name: Integration Test with Localestia |
Copilot is powered by AI and may make mistakes. Always verify output.
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds comprehensive CI/CD infrastructure and testing components for the op-alt-da project. The changes introduce automated workflows for testing, building, releasing, and maintaining the project, along with supporting scripts and configuration files.
- Adds GitHub Actions workflows for CI, integration testing, releases, security scanning, and package cleanup
- Introduces Docker configuration and development tooling via Makefile, docker-compose, and helper scripts
- Sets up PR/issue templates and monitoring infrastructure with Prometheus
- Refactors function naming from public to private (SubmitAndCreateBlobID → submitAndCreateBlobID)
- Updates import path from go-square/v2 to v3 and moves dependency from indirect to direct
Reviewed Changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| setup.sh | Setup script for automating local development environment configuration |
| scripts/ghcr-auth.sh | Helper script for GitHub Container Registry authentication |
| monitoring/prometheus.yml | Prometheus configuration for metrics collection from DA server and localestia |
| go.mod | Updates go-square dependency from v2 to v3 and promotes it to direct dependency |
| docker-compose.yml | Defines local development environment with localestia, DA server, and optional monitoring/S3 services |
| cmd/daserver/main_test.go | Adds placeholder integration and benchmark test structure |
| celestia_storage.go | Updates function call to use new private function name |
| celestia_server.go | Updates import path to use go-square v3 |
| Makefile | Comprehensive build system with development, testing, and Docker targets |
| Dockerfile | Multi-stage build with security improvements and proper metadata |
| .github/workflows/release copy.yml | Release workflow for building and publishing binaries and Docker images |
| .github/workflows/integration.yml | Integration testing workflows with localestia and Kurtosis |
| .github/workflows/ghcr-cleanup.yml | Automated cleanup of old container images in GHCR |
| .github/workflows/dependency-review.yml | Dependency security review for pull requests |
| .github/workflows/codeql.yml | CodeQL security analysis workflow |
| .github/workflows/ci.yml | Main CI pipeline with linting, testing, building, and security scanning |
| .github/pull_request_template.md | Standardized PR template for consistent submissions |
| .github/ISSUE_TEMPLATE/feature_request.md | Template for feature requests |
| .github/ISSUE_TEMPLATE/bug_report.md | Template for bug reports |
Comments suppressed due to low confidence (1)
go.mod:3
- Go version 1.24.6 does not exist. As of January 2025, the latest stable Go version is 1.22.x. Update this to a valid Go version such as '1.21' or '1.22'.
go 1.24.6
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+8
to
9
| RUN make da-server | ||
|
|
There was a problem hiding this comment.
The Dockerfile copies files and sets up the workspace after line 8, but attempts to run 'make da-server' at line 8 before the source code is copied. This will fail because the source files don't exist yet. These lines should be removed as they are superseded by the complete build process defined later in the file (lines 16-33).
Suggested change
| RUN make da-server |
Comment on lines
+96
to
+98
| _, cancel := context.WithTimeout(context.Background(), 30*time.Second) | ||
| defer cancel() | ||
|
|
There was a problem hiding this comment.
The context variable is assigned but never used. Either use the context in the test implementation or remove the underscore: 'ctx, cancel := context.WithTimeout(...)'
Suggested change
| _, cancel := context.WithTimeout(context.Background(), 30*time.Second) | |
| defer cancel() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Adds workflows for testing and releasing new versions