Skip to content
71 changes: 71 additions & 0 deletions docs/book/component-guide/alerters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,76 @@ zenml stack register ... -al <ALERTER_NAME>
Afterward, you can import the alerter standard steps provided by the respective integration and directly use them in
your pipelines.

## Using the Ask Step for Human-in-the-Loop Workflows

All alerters provide an `ask()` method and corresponding ask steps that enable human-in-the-loop workflows. These are essential for:

- Getting approval before deploying models to production
- Confirming critical pipeline decisions
- Manual intervention points in automated workflows

### How Ask Steps Work

Ask steps (like `discord_alerter_ask_step` and `slack_alerter_ask_step`):

1. **Post a message** to your chat service with your question
2. **Wait for user response** containing specific approval or disapproval keywords
3. **Return a boolean** - `True` if approved, `False` if disapproved or timeout

```python
from zenml import step, pipeline
from zenml.integrations.slack.steps.slack_alerter_ask_step import slack_alerter_ask_step

@step
def deploy_model(model, approved: bool) -> None:
if approved:
# Deploy the model to production
print("Deploying model to production...")
# deployment logic here
else:
print("Deployment cancelled by user")

@pipeline
def deployment_pipeline():
trained_model = train_model()
# Ask for human approval before deployment
approved = slack_alerter_ask_step("Deploy model to production?")
deploy_model(trained_model, approved)
```

### Default Response Keywords

By default, alerters recognize these response options:

**Approval:** `approve`, `LGTM`, `ok`, `yes`
**Disapproval:** `decline`, `disapprove`, `no`, `reject`

### Customizing Response Keywords

You can customize the approval and disapproval keywords using alerter parameters:

```python
from zenml.integrations.slack.steps.slack_alerter_ask_step import slack_alerter_ask_step
from zenml.integrations.slack.alerters.slack_alerter import SlackAlerterParameters

# Use custom approval/disapproval keywords
params = SlackAlerterParameters(
approve_msg_options=["deploy", "ship it", "✅"],
disapprove_msg_options=["stop", "cancel", "❌"]
)

approved = slack_alerter_ask_step(
"Deploy model to production?",
params=params
)
```

### Important Notes

- **Return Type**: Ask steps return a boolean value - ensure your pipeline logic handles this correctly
- **Keywords**: Response keywords are case-sensitive (except Slack, which converts to lowercase)
- **Timeout**: If no valid response is received within the timeout period, the step returns `False`
- **Permissions**: Ensure your bot has permissions to read messages in the target channel

<!-- For scarf -->
<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>
81 changes: 73 additions & 8 deletions docs/book/component-guide/alerters/custom.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,13 @@ The base abstraction for alerters is very basic, as it only defines two abstract
* `post()` takes a string, posts it to the desired chat service, and returns `True` if the operation succeeded, else `False`.
* `ask()` does the same as `post()`, but after sending the message, it waits until someone approves or rejects the operation from within the chat service (e.g., by sending "approve" / "reject" to the bot as a response). `ask()` then only returns `True` if the operation succeeded and was approved, else `False`.

The `ask()` method is particularly useful for implementing human-in-the-loop workflows. When implementing this method, you should:
- Wait for user responses containing approval keywords (like `"approve"`, `"yes"`, `"ok"`, `"LGTM"`)
- Wait for user responses containing disapproval keywords (like `"reject"`, `"no"`, `"cancel"`, `"stop"`)
- Return `True` only when explicit approval is received
- Return `False` for disapproval, timeout, or any errors
- Consider implementing configurable approval/disapproval keywords via parameters

Then base abstraction looks something like this:

```python
Expand All @@ -40,7 +47,7 @@ This is a slimmed-down version of the base implementation. To see the full docst

### Building your own custom alerter

Creating your own custom alerter can be done in three steps:
Creating your own custom alerter can be done in four steps:

1. Create a class that inherits from the `BaseAlerter` and implement the `post()` and `ask()` methods.

Expand All @@ -54,18 +61,58 @@ Creating your own custom alerter can be done in three steps:
"""My alerter class."""

def post(
self, message: str, config: Optional[BaseAlerterStepParameters]
self, message: str, params: Optional[BaseAlerterStepParameters]
) -> bool:
"""Post a message to a chat service."""
...
return "Hey, I implemented an alerter."
try:
# Implement your chat service posting logic here
# e.g., send HTTP request to chat API
logging.info(f"Posting message: {message}")
return True
except Exception as e:
logging.error(f"Failed to post message: {e}")
return False

def ask(
self, question: str, config: Optional[BaseAlerterStepParameters]
self, question: str, params: Optional[BaseAlerterStepParameters]
) -> bool:
"""Post a message to a chat service and wait for approval."""
...
return True
try:
# First, post the question
if not self.post(question, params):
return False

# Define default approval/disapproval options
approve_options = ["approve", "yes", "ok", "LGTM"]
disapprove_options = ["reject", "no", "cancel", "stop"]

# Check if custom options are provided in params
if params and hasattr(params, 'approve_msg_options'):
approve_options = params.approve_msg_options
if params and hasattr(params, 'disapprove_msg_options'):
disapprove_options = params.disapprove_msg_options

# Wait for response (implement your chat service polling logic)
# This is a simplified example - you'd implement actual polling
response = self._wait_for_user_response()

if response.lower() in [opt.lower() for opt in approve_options]:
return True
elif response.lower() in [opt.lower() for opt in disapprove_options]:
return False
else:
# Invalid response or timeout
return False

except Exception as e:
print(f"Failed to get approval: {e}")
return False

def _wait_for_user_response(self) -> str:
"""Wait for user response - implement based on your chat service."""
# This is where you'd implement the actual waiting logic
# e.g., polling your chat service API for new messages
return "approve" # Placeholder
```
2. If you need to configure your custom alerter, you can also implement a config object.

Expand All @@ -76,7 +123,25 @@ Creating your own custom alerter can be done in three steps:
class MyAlerterConfig(BaseAlerterConfig):
my_param: str
```
3. Finally, you can bring the implementation and the configuration together in a new flavor object.

3. Optionally, you can create custom parameter classes to support configurable approval/disapproval keywords:

```python
from typing import List, Optional
from zenml.alerter.base_alerter import BaseAlerterStepParameters


class MyAlerterParameters(BaseAlerterStepParameters):
"""Custom parameters for MyAlerter."""

# Custom approval/disapproval message options
approve_msg_options: Optional[List[str]] = None
disapprove_msg_options: Optional[List[str]] = None

# Any other custom parameters for your alerter
custom_channel: Optional[str] = None
```
4. Finally, you can bring the implementation and the configuration together in a new flavor object.

```python
from typing import Type, TYPE_CHECKING
Expand Down
65 changes: 64 additions & 1 deletion docs/book/component-guide/alerters/discord.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,23 @@ zenml alerter register discord_alerter \
--default_discord_channel_id=<DISCORD_CHANNEL_ID>
```

{% hint style="info" %}
**Using Secrets for Token Management**: Instead of passing your Discord token directly, it's recommended to store it as a ZenML secret and reference it in your alerter configuration. This approach keeps sensitive information secure:

```shell
# Create a secret for your Discord token
zenml secret create discord_secret --discord_token=<DISCORD_TOKEN>

# Register the alerter referencing the secret
zenml alerter register discord_alerter \
--flavor=discord \
--discord_token={{discord_secret.discord_token}} \
--default_discord_channel_id=<DISCORD_CHANNEL_ID>
```

Learn more about [referencing secrets in stack component attributes and settings](https://docs.zenml.io/concepts/secrets#reference-secrets-in-stack-component-attributes-and-settings).
{% endhint %}

After you have registered the `discord_alerter`, you can add it to your stack like this:

```shell
Expand Down Expand Up @@ -97,18 +114,64 @@ def my_formatter_step(artifact_to_be_communicated) -> str:
return f"Here is my artifact {artifact_to_be_communicated}!"


@step
def conditional_step(artifact, approved: bool) -> None:
if approved:
# Proceed with the operation
print(f"User approved! Processing {artifact}")
# Your logic here
else:
print("User disapproved. Skipping operation.")


@pipeline
def my_pipeline(...):
...
artifact_to_be_communicated = ...
message = my_formatter_step(artifact_to_be_communicated)
approved = discord_alerter_ask_step(message)
... # Potentially have different behavior in subsequent steps if `approved`
conditional_step(artifact_to_be_communicated, approved)

if __name__ == "__main__":
my_pipeline()
```

## Using Custom Approval Keywords

You can customize which words trigger approval or disapproval by using `DiscordAlerterParameters`:

```python
from zenml.integrations.discord.steps.discord_alerter_ask_step import discord_alerter_ask_step
from zenml.integrations.discord.alerters.discord_alerter import DiscordAlerterParameters

# Custom approval/disapproval keywords
params = DiscordAlerterParameters(
approve_msg_options=["deploy", "ship it", "✅"],
disapprove_msg_options=["stop", "cancel", "❌"]
)

approved = discord_alerter_ask_step(
"Deploy model to production?",
params=params
)
```

### Default Response Keywords

By default, the Discord alerter recognizes these keywords:

**Approval:** `approve`, `LGTM`, `ok`, `yes`
**Disapproval:** `decline`, `disapprove`, `no`, `reject`

**Important Notes:**
- The ask step returns a boolean (`True` for approval, `False` for disapproval/timeout)
- **Keywords are case-sensitive** - you must respond with exact case (e.g., `LGTM` not `lgtm`)
- If no valid response is received, the step returns `False`

{% hint style="warning" %}
**Discord Case Sensitivity**: The Discord alerter implementation requires exact case matching for approval keywords. Make sure to respond with the exact case specified (e.g., `LGTM`, not `lgtm`).
{% endhint %}

For more information and a full list of configurable attributes of the Discord alerter, check out the [SDK Docs](https://sdkdocs.zenml.io/latest/integration_code_docs/integrations-discord.html#zenml.integrations.discord) .

<figure><img src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" alt="ZenML Scarf"><figcaption></figcaption></figure>
47 changes: 43 additions & 4 deletions docs/book/component-guide/alerters/slack.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,12 @@ zenml alerter register slack_alerter \
--slack_channel_id=<SLACK_CHANNEL_ID>
```

{% hint style="info" %}
**Using Secrets for Token Management**: The example above demonstrates the recommended approach of storing your Slack token as a ZenML secret and referencing it using the `{{secret_name.key}}` syntax. This keeps sensitive information secure and follows security best practices.

Learn more about [referencing secrets in stack component attributes and settings](https://docs.zenml.io/concepts/secrets#reference-secrets-in-stack-component-attributes-and-settings).
{% endhint %}

Here is where you can find the required parameters:

* `<SLACK_CHANNEL_ID>`: The channel ID can be found in the channel details.
Expand Down Expand Up @@ -171,7 +177,7 @@ def post_statement() -> None:
)


# Formatting with blocks
# Formatting with blocks and custom approval options
@step
def ask_question() -> bool:
message = ":tada: Should I continue? (Y/N)"
Expand All @@ -193,11 +199,20 @@ def ask_question() -> bool:
)
return Client().active_stack.alerter.ask(question=message, params=params)

@step
def conditional_step(approved: bool) -> None:
if approved:
print("User approved! Continuing with operation...")
# Your logic here
else:
print("User declined. Stopping operation.")


@pipeline(enable_cache=False)
def my_pipeline():
post_statement()
ask_question()
approved = ask_question()
conditional_step(approved)


if __name__ == "__main__":
Expand All @@ -211,25 +226,49 @@ If you want to only use it in a simple manner, you can also use the steps
the Slack integration of ZenML:

```python
from zenml import pipeline
from zenml import pipeline, step
from zenml.integrations.slack.steps.slack_alerter_post_step import (
slack_alerter_post_step
)
from zenml.integrations.slack.steps.slack_alerter_ask_step import (
slack_alerter_ask_step,
)

@step
def handle_response(approved: bool) -> None:
if approved:
print("Operation approved!")
else:
print("Operation declined.")

@pipeline(enable_cache=False)
def my_pipeline():
slack_alerter_post_step("Posting a statement.")
slack_alerter_ask_step("Asking a question. Should I continue?")
approved = slack_alerter_ask_step("Asking a question. Should I continue?")
handle_response(approved)


if __name__ == "__main__":
my_pipeline()
```

## Default Response Keywords and Ask Step Behavior

The `ask()` method and `slack_alerter_ask_step` recognize these keywords by default:

**Approval:** `approve`, `LGTM`, `ok`, `yes`
**Disapproval:** `decline`, `disapprove`, `no`, `reject`

**Important Notes:**
- The ask step returns a boolean (`True` for approval, `False` for disapproval/timeout)
- **Response keywords are case-insensitive** - keywords are converted to lowercase before matching (e.g., both `LGTM` and `lgtm` work)
- If no valid response is received within the timeout period, the step returns `False`
- The default timeout is 300 seconds (5 minutes) but can be configured

{% hint style="info" %}
**Slack Case Handling**: The Slack alerter implementation automatically converts all response keywords to lowercase before matching, making responses case-insensitive. You can respond with `LGTM`, `lgtm`, or `Lgtm` - they'll all work.
{% endhint %}

For more information and a full list of configurable attributes of the Slack
alerter, check out the [SDK Docs](https://sdkdocs.zenml.io/latest/integration_code_docs/integrations-slack.html#zenml.integrations.slack) .

Expand Down