Skip to content

chore: add Github live sync integration tests #542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FelixTJDietrich merged 7 commits into from
Nov 9, 2025
Merged

chore: add Github live sync integration tests #542

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
41192cf
chore: add github integration test
FelixTJDietrich Nov 7, 2025
3e4d64e
Merge branch 'develop' into chore/add-sync-github-integration-test
FelixTJDietrich Nov 7, 2025
2d3b7d0
update docs
FelixTJDietrich Nov 7, 2025
c53085a
update
FelixTJDietrich Nov 7, 2025
39cc6dc
fix ci
FelixTJDietrich Nov 7, 2025
18601e2
fix reflection
FelixTJDietrich Nov 9, 2025
16d9a9f
final changes
FelixTJDietrich Nov 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ venv/
ENV/
env.bak/
venv.bak/
/server/application-server/src/test/resources/application-github-integration-local.yml

# Spyder project settings
.spyderproject
Expand Down
14 changes: 13 additions & 1 deletion AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,15 @@
This file governs the entire repository. Combine these guardrails with the scoped instructions under `.github/instructions/**` (general coding, TSX, Storybook, Java tests).

## 1. Architecture map

- `server/application-server/`: Spring Boot 3.5, Liquibase-managed PostgreSQL schema, synchronous + reactive APIs, generated OpenAPI spec in `openapi.yaml`.
- `webapp/`: React 19 + TanStack Router/Query, Tailwind 4 UI kit (`src/components/ui`), generated API client in `src/api/**`.
- `server/intelligence-service/`: FastAPI service orchestrating AI models. OpenAPI spec is exported via Poetry and mirrored into the Java client under the application server.
- `server/webhook-ingest/`: FastAPI webhook intake that forwards events into NATS JetStream.
- `docs/`: Contributor docs (including the ERD that `db:generate-erd-docs` regenerates).

## 2. Toolchain & environment prerequisites

- **Node.js**: Use the exact version from `.node-version` (currently 22.10.0). Stick with npm—the repo maintains `package-lock.json` and uses npm workspaces.
- **Java**: JDK 21 (see `pom.xml`). Maven wrapper is checked in; **always run builds through `./mvnw`** (Maven wrapper) to ensure consistent Maven versions.
- **Python**: Python 3.13 with Poetry 2.x. Both Python services keep virtualenvs inside their folders (`.venv`). Run `npm run bootstrap:py` before formatting/linting to ensure dev dependencies are installed.
Expand All @@ -18,6 +20,7 @@ This file governs the entire repository. Combine these guardrails with the scope
- **Environment variables**: When generating intelligence service OpenAPI specs locally, set `MODEL_NAME=fake:model` and `DETECTION_MODEL_NAME=fake:model` (the FastAPI settings expect a provider-qualified model name).

## 3. Quality gates & routine commands

Run the relevant commands locally before opening a PR:

| Concern | Commands |
Expand All @@ -27,13 +30,14 @@ Run the relevant commands locally before opening a PR:
| Webapp build | `npm --workspace webapp run build` (Vite build + `tsc --noEmit`) |
| Webapp tests | `npm --workspace webapp run test` (Vitest) and add focused unit tests when touching logic. |
| Storybook | `npm --workspace webapp run build-storybook` (Chromatic depends on a clean build). |
| Application-server tests | Use Maven groups to mirror CI: `./mvnw test -Dgroups=unit`, `-Dgroups=integration`, `-Dgroups=architecture`. |
| Application-server tests | Use Maven groups to mirror CI: `./mvnw test -Dgroups=unit`, `-Dgroups=integration`, `-Dgroups=architecture`. Live GitHub sync tests stay skipped unless you pass `-Dgroups=github-integration`, which activates the `github-integration-tests` profile. |
| Intelligence service lint/type check | `poetry run black --check .`, `poetry run flake8 .`, `poetry run mypy .` inside `server/intelligence-service`. |
| Webhook ingest lint | `poetry run black --check .` and `poetry run flake8 .` inside `server/webhook-ingest`. |

Document any skipped gate in the PR description with a rationale. Always finish a change set by running `npm run format` followed by `npm run lint` so both styling and type checks reflect the final state.

## 4. Code generation & forbidden edits

We rely heavily on generated artifacts. Never hand-edit these directories—regenerate instead:

| Artifact | Source command |
Expand All @@ -48,6 +52,7 @@ We rely heavily on generated artifacts. Never hand-edit these directories—rege
Regeneration is destructive; stash local edits before running these commands. Check diffs carefully—generated clients must be committed alongside API changes.

## 5. Database workflow (Liquibase)

- Liquibase changelog files live under `server/application-server/src/main/resources/db/changelog/` and are included via `master.xml`.
- Use `npm run db:draft-changelog` after changing JPA entities. The script will:
1. Spin up PostgreSQL through Docker (ensure Docker is running or set `CI=true` with a ready Postgres).
Expand All @@ -58,6 +63,7 @@ Regeneration is destructive; stash local edits before running these commands. Ch
- Never manually edit generated Liquibase diff sections unless you fully understand the implications. Prefer creating a follow-up changelog to fix mistakes.

## 6. Frontend (webapp) expectations

- Follow the container/presentation split already in place (route files under `src/routes/**` fetch data and pass it to components under `src/components/**`). Keep components pure and side-effect free.
- Fetch data exclusively with TanStack Query v5 and the generated helpers in `@/api/@tanstack/react-query.gen.ts`. Spread the option objects: `useQuery(getTeamsOptions({ ... }))`. Use the generated `*.QueryKey()` helpers for cache invalidation.
- Do not call `fetch` directly; reuse the generated `@hey-api` client configured in `src/api/client.ts` and the shared QueryClient from `src/integrations/tanstack-query/root-provider.tsx`.
Expand All @@ -70,6 +76,7 @@ Regeneration is destructive; stash local edits before running these commands. Ch
- Never hand-edit `routeTree.gen.ts`; it is generated by TanStack Router tooling.

## 7. Application server (Java/Spring) expectations

- Keep business logic in services annotated with `@Service` and transactional boundaries (`@Transactional`) where needed. Controllers should be thin (input validation + delegation).
- Use Lombok consistently (`@Getter`, `@Setter`, etc.) but prefer explicit builders or records when immutability helps.
- Group new tests under the proper JUnit tag so CI picks them up (`@Tag("unit")`, `@Tag("integration")`, or `@Tag("architecture")`). Follow the mantra in `.github/instructions/java-tests.instructions.md` (AAA structure, single assertion focus, deterministic data).
Expand All @@ -80,6 +87,7 @@ Regeneration is destructive; stash local edits before running these commands. Ch
- When integrating with the intelligence-service client, always regenerate (`npm run generate:api:intelligence-service:client`) after touching the spec and commit the updated Java files.

## 8. Python services expectations

- Both services rely on Poetry with in-project virtualenvs. Run `poetry install --with dev --no-root` before running tooling.
- Intelligence service:
- Settings live in `app/settings.py`; `MODEL_NAME` and `DETECTION_MODEL_NAME` must be provider-qualified (`openai:gpt-4o`, `fake:model`, etc.). For tooling/CI we rely on the `fake` provider.
Expand All @@ -91,19 +99,23 @@ Regeneration is destructive; stash local edits before running these commands. Ch
- Formatting: run `poetry run black .` (or `--check`) and `poetry run flake8 .`. Add type hints so mypy stays green in the intelligence service.

## 9. Documentation & assets

- ERD diagrams live under `docs/contributor/erd/`. Regenerate via `npm run db:generate-erd-docs` after schema changes.
- Contributor documentation should stay in `docs/` (GitHub Pages). Keep README/CONTRIBUTING updates concise and actionable.
- Screenshots or large binary assets belong under `docs/images/` or the Storybook stories, not inside source directories.

## 10. Commit & PR checklist

Before marking work ready for review:

- [ ] Regenerate and commit any impacted OpenAPI specs, clients, ERD docs, or generated SQLAlchemy models.
- [ ] Run the formatting/linting/typecheck/test commands relevant to the modified modules; capture output for the PR description if CI cannot run a job locally.
- [ ] Verify database migrations through `db:draft-changelog` when JPA entities change and inspect the produced XML.
- [ ] Double-check that no generated files were edited manually.
- [ ] Follow Conventional Commit semantics for PR titles (`feat(webapp): ...`, etc., see `CONTRIBUTING.md`).

## 11. Known command caveats

- `npm run db:draft-changelog` requires Docker to be installed and available on PATH. In CI we set `CI=true`; locally ensure Docker Desktop/daemon is running before invoking the script.
- `npm run generate:api:intelligence-service:specs` fails unless `MODEL_NAME` and `DETECTION_MODEL_NAME` are set (use the `fake:model` provider for tooling).
- `npm run generate:api:application-server:specs` performs a full Maven `verify` against the specs profile. The initial run downloads the entire Spring Boot dependency tree (~hundreds of MB); expect several minutes on a cold cache.
Expand Down
57 changes: 57 additions & 0 deletions docs/contributor/testing.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,3 +96,60 @@ Available examples include `label.created`, `repository.created`, `create`, `pus

Use the optional `-Dgroups=unit` or `-Dgroups=integration` flags once category support lands.

## GitHub live sync integration tests

Some regression scenarios can only be validated against GitHub itself. We ship a focused suite that exercises the live GitHub App installation and verifies end-to-end sync behaviour (repository metadata, labels, milestones, and teams).

### Prerequisites

1. **Sandbox installation** – the `Hephaestus IntegrationTests` GitHub App must be installed in a sandbox organisation you control. The tests create and delete repositories, milestones, labels, and teams on each run.
2. **Credentials** – provide both a GitHub App private key and a Personal Access Token with the following scopes:
- `repo` (full)
- `admin:org`
- `read:packages`
3. **Local config file** – copy the template that lives alongside the tests:

```bash
cd server/application-server/src/test/resources
cp application-github-integration-local.example.yml application-github-integration-local.yml
```

Fill in the placeholders with the sandbox organisation slug, the installation id, and either an inline PEM key (`github.app.privateKey`) or a readable `privateKeyLocation`. Keep this file out of version control—it is already listed in `.gitignore`.
Comment on lines +113 to +117
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Nov 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Verify configuration property names match implementation.

The section references an inline github.app.privateKey property and privateKeyLocation. Confirm these property names match the actual application configuration schema and that the documentation accurately reflects supported configuration options.

🏁 Script executed:

#!/bin/bash
# Description: Verify GitHub app property names in configuration

# Search for property definitions and usage
rg "github\.app\.privateKey|privateKeyLocation" --type=java --type=yaml -B 2 -A 2 | head -50

Length of output: 5687

Update documentation to use the correct property name github.app.privateKeyLocation.

The documentation references privateKeyLocation, but the actual configuration property is github.app.privateKeyLocation. Update line 117 to use the full qualified property name for consistency with the inline github.app.privateKey reference.

🧰 Tools
🪛 LanguageTool

[uncategorized] ~117-~117: The official name of this software platform is spelled with a capital “H”.
Context: ...ation id, and either an inline PEM key (github.app.privateKey) or a readable `private...

(GITHUB)

🤖 Prompt for AI Agents 
In docs/contributor/testing.mdx around lines 113 to 117, the docs mention the
config key `privateKeyLocation` but the correct fully-qualified property is
`github.app.privateKeyLocation`; update the text on line 117 to reference
`github.app.privateKeyLocation` (so it matches the inline
`github.app.privateKey` reference) and ensure the example and explanatory
sentence use the fully qualified name.


Alternatively, export the matching environment variables:

```bash
export GH_IT_APP_ID=2250297
export GH_IT_APP_PAT=ghp_xxx... # PAT with the scopes above
export GH_IT_INSTALLATION_ID=93512943
export GH_IT_ORGANIZATION=HephaestusTest
export GH_IT_APP_PRIVATE_KEY_PATH=/absolute/path/to/private-key.pem
```

Use either the file _or_ the environment variables; the suite checks both and aborts if key material is missing.

### Running the suite

From `server/application-server/` run:

```bash
./mvnw test -Dgroups=github-integration
```

Passing the `groups` system property automatically activates the `github-integration-tests` Maven profile, which clears the default exclusion and runs only the live GitHub scenarios. The regular CI pipeline never sets this flag, so these tests remain skipped unless you opt in locally.

The run takes roughly two minutes and prints the GitHub artefacts it provisions. Clean-up is handled automatically, but if a failure interrupts execution you can safely delete any `hephaestus-it-*` repositories, milestones, or teams that remain in the sandbox.

### Authoring new GitHub sync tests

1. Extend `AbstractGitHubSyncIntegrationTest` (or fall back to `BaseGitHubIntegrationTest` when repositories are not needed). These bases set up credential checks, provide the `workspaceRepository`, and expose helpers such as `createEphemeralRepository`, `registerRepositoryToMonitor`, `createEphemeralTeam`, and `seedOrganizationMembers`.
2. Use the supplied helpers to create and track temporary GitHub artefacts. They automatically register clean-up handlers via `@AfterEach`, so add new resources to the provided lists instead of implementing manual deletion logic.
3. Keep tests deterministic: rely on `databaseTestUtils.cleanDatabase()` in `@BeforeEach` (already invoked by the base), and generate unique slugs via `nextEphemeralSlug("suffix")` when naming repositories, branches, or teams.
4. If a scenario needs extra Spring configuration, extend `application-github-integration-local.yml` in `server/application-server/src/test/resources/`. The checked-in `.example` file documents every property; copy it on demand and keep secrets out of version control.

### Troubleshooting

- **Skipping because of missing credentials** – check the console output; the base test class verifies that the App id, private key, PAT, and installation id are all present before executing.
- **Hub4j rate-limit failures** – the suite creates several entities per run. Prefer a dedicated sandbox organisation so you do not clash with production automation limits.
- **Longer runtimes** – each suite bootstraps a Testcontainers PostgreSQL instance and provisions GitHub resources. Expect higher runtimes than the pure Testcontainers integration tests; avoid running them on every PR and instead use them before releases or when touching the GitHub sync layer.

Loading