-
Notifications
You must be signed in to change notification settings - Fork 439
chore(telemetry): report tested integration versions #6853
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Although this approach currently works it uses the instrumentation telemetry client in an unsupported way. To support python3.12 we recently changed when/how telemtry events are sent. This change will prevent integrations will not be sent in test runs: #6859. We can hack around this issue but I don't think this is the best approach.
Can we get the tested integration versions by parsing the venvs in https://github.com/DataDog/dd-trace-py/blob/1.x/riotfile.py? This way we can collect and submit the integration data to the test agent once. We can also make this nightly job on the 1.x branch so this data is not collected on every ci run.
BenchmarksBenchmark execution time: 2023-10-16 14:18:22 Comparing candidate commit b2d4ce4 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 90 metrics, 0 unstable metrics. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is beyond the scope of this PR but some of the modules have the name test_integrationname_patch_generated
. We need to remove the generated part of the name and make sure the patch test is actually making the expected assertions.
…:DataDog/dd-trace-py into conti/test-integration-versions-branch-2
Motivation
With this PR we want to add a method to the generated patch for all contribs that will emit a telemetry payload describing the tested integration and its tested version. The goal of this PR is to get a list of all integrations tested during CI, along with the range of versions tested, which will be reported to metabase. The reasoning for adding another testcase instead of simply using information available in metabase from customer telemetry on what integrations are used is as follows:
Checklist
changelog/no-changelog
.Reviewer Checklist
@DataDog/security-design-and-guidance
.