Skip to content

Enterprise multi-node recommendations and Python libraries update #6107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Jun 9, 2025
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
8897471
docs(enterprise): scope distributed cluster guidance to Enterprise-on…
MeelahMe May 29, 2025
1ac1c8d
Merge branch 'master' into docs/generalize-grafana-multinode
MeelahMe May 29, 2025
6db9b4d
docs(pe): add tabbed examples and warning for installing Python packages
MeelahMe May 29, 2025
aa97879
Merge branch 'docs/generalize-grafana-multinode' of github.com:influx…
MeelahMe May 29, 2025
66b39d3
minor inlin1e code updates
MeelahMe May 29, 2025
877d263
Update content/shared/v3-core-plugins/_index.md
MeelahMe May 29, 2025
3863ebc
Update content/shared/v3-core-plugins/_index.md
MeelahMe May 29, 2025
5f7fe5f
Rewrote install section to improve flow and clarify when to use CLI v…
MeelahMe May 29, 2025
3a798fd
Merge branch 'docs/generalize-grafana-multinode' of github.com:influx…
MeelahMe May 29, 2025
a336481
Merge branch 'master' into docs/generalize-grafana-multinode
MeelahMe May 29, 2025
25ad299
Update content/shared/v3-core-plugins/_index.md
MeelahMe Jun 9, 2025
d384d87
Update content/shared/v3-core-plugins/_index.md
MeelahMe Jun 9, 2025
1444876
Update content/shared/v3-core-plugins/_index.md
MeelahMe Jun 9, 2025
6283517
Update content/shared/v3-core-plugins/_index.md
MeelahMe Jun 9, 2025
9e3ab82
Update content/shared/v3-core-plugins/_index.md
MeelahMe Jun 9, 2025
5b3b4a8
Update content/shared/v3-core-plugins/_index.md
MeelahMe Jun 9, 2025
efaf77c
Update content/shared/v3-core-plugins/_index.md
MeelahMe Jun 9, 2025
b91612f
Merge branch 'master' into docs/generalize-grafana-multinode
MeelahMe Jun 9, 2025
bc7289a
Merge branch 'master' into docs/generalize-grafana-multinode
MeelahMe Jun 9, 2025
c142f64
Merge branch 'master' into docs/generalize-grafana-multinode
jstirnaman Jun 9, 2025
af09992
Update content/shared/v3-core-plugins/_index.md
MeelahMe Jun 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions assets/jsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@
"baseUrl": ".",
"paths": {
"*": [
"*",
"../node_modules/*"
"*"
]
}
}
Expand Down
79 changes: 71 additions & 8 deletions content/shared/v3-core-plugins/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -523,27 +523,90 @@ influxdb3 create trigger \

### Install Python dependencies

If your plugin needs additional Python packages, use the `influxdb3 install` command:
Use the `influxdb3 install package` command to add third-party libraries (like `pandas`, `requests`, or `influxdb3-python`) to your plugin environment.
This installs packages into the Processing Engine’s embedded Python environment to ensure compatibility with your InfluxDB instance.
Comment on lines +526 to +527
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥇


{{% code-placeholders "CONTAINER_NAME|PACKAGE_NAME" %}}

{{< code-tabs-wrapper >}}

{{% code-tabs %}}
[CLI](#)
[Docker](#)
{{% /code-tabs %}}

{{% code-tab-content %}}

```bash
# Install a package directly
# Use the CLI to install a Python package
influxdb3 install package pandas

```

{{% /code-tab-content %}}

{{% code-tab-content %}}

```bash
# With Docker
# Use the CLI to install a Python package in a Docker container
docker exec -it CONTAINER_NAME influxdb3 install package pandas
```

This creates a Python virtual environment in your plugins directory with the specified packages installed.
{{% /code-tab-content %}}

{{< /code-tabs-wrapper >}}

These examples install the specified Python package (for example, pandas) into the Processing Engine’s embedded virtual environment.

- Use the CLI command when running InfluxDB directly on your system.
- Use the Docker variant if you're running InfluxDB in a containerized environment.

> [!Important]
> #### Use bundled Python for plugins
> When you start the server with the `--plugin-dir` option, InfluxDB 3 creates a Python virtual environment (`<PLUGIN_DIR>/venv`) for your plugins.
> If you need to create a custom virtual environment, use the Python interpreter bundled with InfluxDB 3—not the system Python.
> Creating a virtual environment with the system Python (for example, using `python -m venv`) can lead to runtime errors and plugin failures.
>
>For more information, see the [processing engine README](https://github.com/influxdata/influxdb/blob/main/README_processing_engine.md#official-builds).

{{% /code-placeholders %}}

InfluxDB creates a Python virtual environment in your plugins directory with the specified packages installed.

{{% show-in "enterprise" %}}

### Connect Grafana to your InfluxDB instance
## Distributed cluster considerations

When you deploy {{% product-name %}} in a multi-node environment, configure each node based on its role and the plugins it runs.

### Match plugin types to the correct node

Each plugin must run on a node that supports its trigger type:

| Plugin type | Trigger spec | Runs on |
|--------------------|--------------------------|-----------------------------|
| Data write | `table:` or `all_tables` | Ingester nodes |
| Scheduled | `every:` or `cron:` | Any node with scheduler |
| HTTP request | `path:` | Nodes that serve API traffic|

For example:
- Run write-ahead log (WAL) plugins on ingester nodes.
Copy link
Preview

Copilot AI May 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The example mentions WAL plugins but the trigger types table lists only data write, scheduled, and HTTP request triggers. Consider aligning the example with the table or explaining how WAL plugins map to the data write category.

Copilot uses AI. Check for mistakes.

- Run scheduled plugins on any node configured to execute them.
- Run HTTP-triggered plugins on querier nodes or any node that handles HTTP endpoints.

Place all plugin files in the `--plugin-dir` directory configured for each node.

> [!Note]
> Triggers fail if the plugin file isn’t available on the node where it runs.

### Route third-party clients to querier nodes

External tools—such as Grafana, custom dashboards, or REST clients—must connect to querier nodes in your InfluxDB Enterprise deployment.

When configuring Grafana to connect to an InfluxDB 3 Enterprise instance:
#### Examples

- **URL**: Use a querier URL or any node that serves queries
- **Grafana**: When adding InfluxDB 3 as a Grafana data source, use a querier node URL, such as:
`https://querier.example.com:8086`
- **REST clients**: Applications using `POST /api/v3/query/sql` or similar endpoints must target a querier node.

Example URL format: `https://querier.your-influxdb.com:8086`
{{% /show-in %}}