feat(py): add telemetry plugins and new model providers (Mistral AI, Hugging Face)#4406
feat(py): add telemetry plugins and new model providers (Mistral AI, Hugging Face)#4406
Conversation
github-actions
bot
added
docs
Summary of ChangesHello @yesudeep, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly expands the Python Genkit framework's observability capabilities by integrating with a wider range of telemetry providers. It introduces new plugins for Azure Application Insights and generic OTLP export (compatible with Cloudflare and others), alongside a versatile observability plugin that supports popular third-party services through pre-configured settings. These additions are accompanied by comprehensive documentation updates and new sample applications, making it easier for developers to monitor their Genkit applications across diverse cloud and observability platforms. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
yesudeep
force-pushed
the
yesudeep/feat/uptight-language
branch
from
cb79c61 to
60dbf87
Compare
yesudeep
changed the title
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces three valuable telemetry plugins for Azure, Cloudflare (and other OTLP backends), and a general observability plugin for popular third-party platforms. The code is well-structured and includes comprehensive documentation and samples. My review focuses on the observability plugin, where I've identified a critical issue with the Sentry configuration and a couple of medium-severity improvement opportunities related to the Grafana configuration and log-trace correlation that would enhance usability and align it with the other new telemetry plugins.
py/plugins/observability/src/genkit/plugins/observability/__init__.py
Outdated
Show resolved
Hide resolved
py/plugins/observability/src/genkit/plugins/observability/__init__.py
Outdated
Show resolved
Hide resolved
yesudeep
force-pushed
the
yesudeep/feat/uptight-language
branch
2 times, most recently
from
1dcbf64 to
b0f1dd7
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This PR is a significant feature addition, introducing three new telemetry plugins (azure, cf, observability) and corresponding samples. The code quality is high, with excellent documentation in docstrings, including diagrams and ELI5 explanations. The plugin design, especially the observability plugin with its backend presets, is well-thought-out and provides a great user experience. The changes to the development guidelines and the new README.md for samples are also valuable additions. I've identified a few areas for improvement related to packaging configuration, test clarity, and the robustness of the Sentry integration. Overall, this is a fantastic contribution.
yesudeep
force-pushed
the
yesudeep/feat/uptight-language
branch
6 times, most recently
from
5c4bd58 to
5f4fdf4
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This is an impressive and substantial contribution. You've added three powerful telemetry plugins (azure, cf, observability) that significantly expand Genkit's observability capabilities. The architecture of these plugins, especially the observability plugin with its backend presets, is well-designed and provides a great user experience. The new samples are clear and will be very helpful for users to get started.
The extensive documentation updates, including the new plugin development guidelines in GEMINI.md and the comprehensive README files, are of very high quality and will be invaluable for the community and future contributors. The attention to detail, like fixing the Mistral model prefix and improving the Grafana auth flow, is also appreciated.
I found one minor inconsistency in the new documentation, which I've commented on. Overall, this is an excellent pull request that greatly enhances the Genkit Python SDK.
yesudeep
force-pushed
the
yesudeep/feat/uptight-language
branch
4 times, most recently
from
95e9ad0 to
4f4c1b6
Compare
yesudeep
changed the title
yesudeep
force-pushed
the
yesudeep/feat/uptight-language
branch
from
4f4c1b6 to
451761b
Compare
yesudeep
force-pushed
the
yesudeep/feat/uptight-language
branch
8 times, most recently
from
761b5f6 to
54d50a4
Compare
yesudeep
force-pushed
the
yesudeep/feat/uptight-language
branch
from
54d50a4 to
8a91a67
Compare
…ramework The new Ruff security (S110, S112, S310) and print (T201) rules require changes to the core framework code. Since the core framework code owner is unavailable for approvals, temporarily ignore these rules for the packages/genkit/ directory. These will be addressed in a separate PR with core framework changes.
|
/gemini review |
… URLs directly The test was trying to use httpx to fetch a data URL, but httpx only supports http:// and https:// protocols. Data URLs are inline base64 encoded data and should be decoded directly. Changed the test to: - Parse the data URL and verify its format - Decode the base64 content directly - Compare with expected bytes
Same fix as the Gemini test - decode data URLs directly instead of trying to fetch them with httpx (which only supports http/https).
There was a problem hiding this comment.
Code Review
This is an impressive pull request that significantly expands the capabilities of the Genkit Python SDK. The addition of new telemetry plugins (Azure, Cloudflare/OTLP, Observability presets) and major model providers (Mistral AI, Hugging Face) is a huge win for users. I'm particularly impressed by the extensive work on release automation and consistency checking, which will greatly improve the long-term maintainability of the repository. The updates to the GEMINI.md contributing guide are thorough and set a high standard for future contributions. The code quality is generally high, with good use of modern Python features and a clear focus on async best practices. My review includes a few minor suggestions for improving code clarity and consistency.
- msfoundry: Sort __all__ exports alphabetically for maintainability - google-genai-code-execution sample: Restore display_code_execution function with proper print statements (samples are allowed to use print for demonstration purposes, with noqa comments)
|
/gemini review |
There was a problem hiding this comment.
Code Review
This is an impressive and substantial pull request that adds significant new functionality with telemetry and model provider plugins. The quality of the new code is very high, with excellent documentation, robust automation scripts, and adherence to best practices. The refactoring of existing code to use modern Python features like pathlib and aiofiles is also a great improvement. I've only found a couple of minor opportunities for improvement related to performance and code consistency.
Replace mixed os.path.join/pathlib.Path usage with pure pathlib operators for better readability and modern Python practices.
yesudeep
force-pushed
the
yesudeep/feat/uptight-language
branch
from
2bb113e to
8c8097a
Compare
2 participants
Summary
This PR introduces three new telemetry plugins and two new model provider plugins for the Python SDK, enabling Genkit users to export observability data to various backends and access additional AI models.
New Plugins
Telemetry Plugins
genkit-plugin-azuregenkit-plugin-cfgenkit-plugin-observabilityModel Provider Plugins (NEW)
genkit-plugin-mistralgenkit-plugin-huggingfacePlugin Architecture
Changes
New Model Provider Plugins (
py/plugins/)mistral/- Mistral AI pluginMistralplugin class with lazy model loadingmistralaiPython SDKMISTRAL_API_KEYhuggingface/- Hugging Face pluginHuggingFaceplugin class with lazy model loadinghuggingface_hubPython SDKHF_TOKENNew Telemetry Plugins (
py/plugins/)azure/- Azure Application Insights telemetry pluginadd_azure_telemetry()function for easy setuptrace_id,span_id)cf/- Cloudflare/OTLP telemetry pluginadd_cf_telemetry()function for OTLP exportobservability/- Third-party observability presetsconfigure_telemetry()with backend presetsNew Samples (
py/samples/)mistral-hello/- Mistral AI demo (chat, streaming, tool calling, structured output)huggingface-hello/- Hugging Face demo (multiple models, streaming, tool calling)azure-hello/- Azure Application Insights democf-hello/- Generic OTLP export demoobservability-hello/- Third-party backend demo (Honeycomb example)Test Coverage
Added comprehensive unit tests for all new plugins:
plugins/mistral/tests/- Plugin and model testsplugins/huggingface/tests/- Plugin and model testsplugins/azure/tests/- Plugin testsplugins/cf/tests/- Plugin testsplugins/observability/tests/- Plugin testsplugins/google-genai/tests/- Plugin testsplugins/firebase/tests/- Plugin testsplugins/evaluators/tests/- Plugin testsplugins/dev-local-vectorstore/tests/- Plugin testsplugins/vertex-ai/tests/- Plugin testsRelease Automation (
py/bin/)check_consistency- Automated consistency checker (18 checks):py.typedmarker files (PEP 561)__all__exports for IDE autocomplete# type: ignorecomments__init__.pyvalidationrelease_check- Comprehensive pre-release validationbump_version- Automated version bumpingfix_classifiers.py- Add missing PyPI classifiersfix_package_metadata.py- Add missing keywords and URLsDocumentation Updates
py/GEMINI.md- Comprehensive updates:bin/lintas mandatory for all PRspy/plugins/README.md- Updated plugin tables with Mistral and Hugging Facepy/samples/README.md- Updated environment variables documentationpy/engdoc/parity-analysis/roadmap.md- Added M8 section for new model provider pluginspy/CHANGELOG.md- Created changelog fileConfiguration Updates
py/pyproject.toml- Added workspace sources for new plugins and samples, added deptry and typos to lint dependencies, temporarily ignored security/async rules for core frameworkbin/lint- Added license check (bin/check_license) and consistency check (py/bin/check_consistency)py/typos.toml- Created typos configuration for spell checkingBug Fixes
plugins/cf-ai/models/model_info.py- Fixed Gemma 7B model prefix (@hf/google/gemma-7b-itinstead of@cf/google/gemma-7b-it)plugins/google-cloud/tests/tracing_test.py- Fixed test to clear project env varsevaluator-demo/package name:eval-demo→evaluator-demomodel-garden/package name:model-garden-example→model-gardenollama-simple-embed/package name:ollama_simple_embed→ollama-simple-embedsamples/evaluator-demo/evaluator_demo/eval_in_code.py- Fixed typo "programatically" → "programmatically"Environment Variables
Model Providers
MISTRAL_API_KEYHF_TOKENTelemetry
SENTRY_DSNHONEYCOMB_API_KEY,HONEYCOMB_DATASET*DD_API_KEY,DD_SITE*GRAFANA_OTLP_ENDPOINT,GRAFANA_USER_ID,GRAFANA_API_KEYAXIOM_TOKEN,AXIOM_DATASET**Optional
Test Plan
bin/lintpasses with 0 errors (with temporary ignores for core framework)bin/check_license)uv run pytest .)__init__.pyingenkit/orgenkit/plugins/)Follow-up Work
The following will be addressed in a separate PR once the core team can review:
S310(urllib.request.urlopen) - replace with httpx async clientS112(try-except-continue) - add explicit noqa comments with justificationS110(try-except-pass) - add explicit noqa comments with justificationT201(print) - replace with logging or warnings.warn