-
Notifications
You must be signed in to change notification settings - Fork 304
Enterprise multi-node recommendations and Python libraries update #6107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 20 commits
8897471
1ac1c8d
6db9b4d
aa97879
66b39d3
877d263
3863ebc
5f7fe5f
3a798fd
a336481
25ad299
d384d87
1444876
6283517
9e3ab82
5b3b4a8
efaf77c
b91612f
bc7289a
c142f64
af09992
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,8 +3,7 @@ | |
"baseUrl": ".", | ||
"paths": { | ||
"*": [ | ||
"*", | ||
"../node_modules/*" | ||
"*" | ||
] | ||
} | ||
} | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -523,27 +523,90 @@ influxdb3 create trigger \ | |
|
||
### Install Python dependencies | ||
|
||
If your plugin needs additional Python packages, use the `influxdb3 install` command: | ||
Use the `influxdb3 install package` command to add third-party libraries (like `pandas`, `requests`, or `influxdb3-python`) to your plugin environment. | ||
This installs packages into the Processing Engine’s embedded Python environment to ensure compatibility with your InfluxDB instance. | ||
|
||
{{% code-placeholders "CONTAINER_NAME|PACKAGE_NAME" %}} | ||
|
||
{{< code-tabs-wrapper >}} | ||
|
||
{{% code-tabs %}} | ||
[CLI](#) | ||
[Docker](#) | ||
{{% /code-tabs %}} | ||
|
||
{{% code-tab-content %}} | ||
|
||
```bash | ||
# Install a package directly | ||
# Use the CLI to install a Python package | ||
influxdb3 install package pandas | ||
|
||
``` | ||
|
||
{{% /code-tab-content %}} | ||
|
||
{{% code-tab-content %}} | ||
|
||
```bash | ||
# With Docker | ||
# Use the CLI to install a Python package in a Docker container | ||
docker exec -it CONTAINER_NAME influxdb3 install package pandas | ||
``` | ||
|
||
This creates a Python virtual environment in your plugins directory with the specified packages installed. | ||
{{% /code-tab-content %}} | ||
|
||
{{< /code-tabs-wrapper >}} | ||
|
||
These examples install the specified Python package (for example, pandas) into the Processing Engine’s embedded virtual environment. | ||
|
||
- Use the CLI command when running InfluxDB directly on your system. | ||
- Use the Docker variant if you're running InfluxDB in a containerized environment. | ||
|
||
> [!Important] | ||
> #### Use bundled Python for plugins | ||
> When you start the server with the `--plugin-dir` option, InfluxDB 3 creates a Python virtual environment (`<PLUGIN_DIR>/venv`) for your plugins. | ||
> If you need to create a custom virtual environment, use the Python interpreter bundled with InfluxDB 3—not the system Python. | ||
MeelahMe marked this conversation as resolved.
Show resolved
Hide resolved
|
||
> Creating a virtual environment with the system Python (for example, using `python -m venv`) can lead to runtime errors and plugin failures. | ||
> | ||
>For more information, see the [processing engine README](https://github.com/influxdata/influxdb/blob/main/README_processing_engine.md#official-builds). | ||
|
||
{{% /code-placeholders %}} | ||
|
||
InfluxDB creates a Python virtual environment in your plugins directory with the specified packages installed. | ||
|
||
{{% show-in "enterprise" %}} | ||
|
||
### Connect Grafana to your InfluxDB instance | ||
## Distributed cluster considerations | ||
|
||
When you deploy {{% product-name %}} in a multi-node environment, configure each node based on its role and the plugins it runs. | ||
|
||
### Match plugin types to the correct node | ||
|
||
Each plugin must run on a node that supports its trigger type: | ||
|
||
| Plugin type | Trigger spec | Runs on | | ||
|--------------------|--------------------------|-----------------------------| | ||
| Data write | `table:` or `all_tables` | Ingester nodes | | ||
| Scheduled | `every:` or `cron:` | Any node with scheduler | | ||
| HTTP request | `path:` | Nodes that serve API traffic| | ||
|
||
For example: | ||
- Run write-ahead log (WAL) plugins on ingester nodes. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [nitpick] The example mentions WAL plugins but the trigger types table lists only data write, scheduled, and HTTP request triggers. Consider aligning the example with the table or explaining how WAL plugins map to the data write category. Copilot uses AI. Check for mistakes. Positive FeedbackNegative Feedback |
||
- Run scheduled plugins on any node configured to execute them. | ||
- Run HTTP-triggered plugins on querier nodes or any node that handles HTTP endpoints. | ||
|
||
Place all plugin files in the `--plugin-dir` directory configured for each node. | ||
|
||
> [!Note] | ||
> Triggers fail if the plugin file isn’t available on the node where it runs. | ||
|
||
### Route third-party clients to querier nodes | ||
|
||
External tools—such as Grafana, custom dashboards, or REST clients—must connect to querier nodes in your InfluxDB Enterprise deployment. | ||
|
||
When configuring Grafana to connect to an InfluxDB 3 Enterprise instance: | ||
#### Examples | ||
|
||
- **URL**: Use a querier URL or any node that serves queries | ||
- **Grafana**: When adding InfluxDB 3 as a Grafana data source, use a querier node URL, such as: | ||
`https://querier.example.com:8086` | ||
- **REST clients**: Applications using `POST /api/v3/query/sql` or similar endpoints must target a querier node. | ||
|
||
Example URL format: `https://querier.your-influxdb.com:8086` | ||
{{% /show-in %}} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🥇