Skip to content

Enable the integration tests for Anthropic #743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 23, 2025
Merged

Enable the integration tests for Anthropic #743

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dd49bb7
Enable the integration tests for Anthropic
rdimitrov Jan 23, 2025
f9f8d83
Add the secret for the anthropic key
rdimitrov Jan 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/integration-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ on:
copilot-key:
description: 'The Copilot key to use for integration tests'
required: true
anthropic-key:
description: 'The Anthropic key to use for integration tests'
required: true

jobs:
integration-tests:
Expand Down Expand Up @@ -136,6 +139,13 @@ jobs:
run: |
poetry run python tests/integration/integration_tests.py

- name: Run integration tests - Anthropic
env:
CODEGATE_PROVIDERS: "anthropic"
ENV_ANTHROPIC_KEY: ${{ secrets.anthropic-key }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we might want to do a matrix in the future, but this is good enough

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about that too, but I wanted to get everything running first and then do that as a follow up 👍 That way they can run in parallel too

run: |
poetry run python tests/integration/integration_tests.py

- name: Print the container logs (useful for debugging)
if: always()
run: |
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/run-on-pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,4 @@ jobs:
artifact-name: "codegate-image"
secrets:
copilot-key: ${{ secrets.INTEGRATION_TESTS_COPILOT_KEY }}
anthropic-key: ${{ secrets.INTEGRATION_TESTS_ANTHROPIC_KEY }}
1 change: 1 addition & 0 deletions .github/workflows/run-on-push.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ jobs:
artifact-name: "codegate-image"
secrets:
copilot-key: ${{ secrets.INTEGRATION_TESTS_COPILOT_KEY }}
anthropic-key: ${{ secrets.INTEGRATION_TESTS_ANTHROPIC_KEY }}
openapi:
name: Generate the OpenAPI docs
needs: [ci]
Expand Down
10 changes: 1 addition & 9 deletions tests/integration/testcases.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,15 +217,7 @@ testcases:
"temperature":0
}
likes: |
Hello! I'm CodeGate, your security-focused AI assistant. I can help you with:

- Software security analysis and reviews
- Package security assessments
- Secure coding practices and guidance
- Security vulnerability analysis
- Best practices for secure implementations

How can I assist you with your security needs today?
Comment on lines -220 to -228
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to change this otherwise it was failing the test. I made it similar to the test we did for Copilot.

Of course if we expect to have a codegate-aware answer let me know as we might have a bug 👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, I think this was a remnant of us adding the codegate prompt to everything

Hello! How can I assist you today?

anthropic_fim:
name: Anthropic FIM
Expand Down
Loading