Skip to content

Commit 2781e4d

Browse files
strickvlclaudeCopilot
authored
Improve alerter documentation with comprehensive ask step coverage (#3693)
* Improve alerter documentation with comprehensive ask step coverage - Add detailed section on ask step functionality to main alerter docs - Document default approval/disapproval keywords (approve, LGTM, ok, yes / decline, disapprove, no, reject) - Show proper ZenML pipeline patterns with conditional logic inside steps - Add custom approval keyword examples for both Discord and Slack - Update custom alerter docs with realistic ask() implementation - Include parameter classes for configurable approval/disapproval options - Document boolean return type and timeout behavior - Add human-in-the-loop workflow examples and best practices 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Update docs/book/component-guide/alerters/custom.md Co-authored-by: Copilot <[email protected]> * Fix case sensitivity documentation to accurately reflect implementation - Clarify that ZenML alerter implementations handle case conversion, not the platforms - Discord alerter: exact case matching in our implementation (discord_alerter.py:300-302) - Slack alerter: lowercase conversion in our implementation (slack_alerter.py:348) - Remove misleading references to platform behavior vs our code behavior 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Add secrets management guidance to Discord and Slack alerter docs - Add info hints in both Discord and Slack alerter documentation - Explain secure token storage using ZenML secrets instead of hardcoding - Include examples showing secret creation and reference syntax - Link to secrets documentation for detailed guidance 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * rename steps --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Copilot <[email protected]>
1 parent 5be0a47 commit 2781e4d

File tree

4 files changed

+251
-13
lines changed

4 files changed

+251
-13
lines changed

‎docs/book/component-guide/alerters/README.md

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,5 +48,76 @@ zenml stack register ... -al <ALERTER_NAME>
4848
Afterward, you can import the alerter standard steps provided by the respective integration and directly use them in
4949
your pipelines.
5050

51+
## Using the Ask Step for Human-in-the-Loop Workflows
52+
53+
All alerters provide an `ask()` method and corresponding ask steps that enable human-in-the-loop workflows. These are essential for:
54+
55+
- Getting approval before deploying models to production
56+
- Confirming critical pipeline decisions
57+
- Manual intervention points in automated workflows
58+
59+
### How Ask Steps Work
60+
61+
Ask steps (like `discord_alerter_ask_step` and `slack_alerter_ask_step`):
62+
63+
1. **Post a message** to your chat service with your question
64+
2. **Wait for user response** containing specific approval or disapproval keywords
65+
3. **Return a boolean** - `True` if approved, `False` if disapproved or timeout
66+
67+
```python
68+
from zenml import step, pipeline
69+
from zenml.integrations.slack.steps.slack_alerter_ask_step import slack_alerter_ask_step
70+
71+
@step
72+
def deploy_model(model, approved: bool) -> None:
73+
if approved:
74+
# Deploy the model to production
75+
print("Deploying model to production...")
76+
# deployment logic here
77+
else:
78+
print("Deployment cancelled by user")
79+
80+
@pipeline
81+
def deployment_pipeline():
82+
trained_model = train_model()
83+
# Ask for human approval before deployment
84+
approved = slack_alerter_ask_step("Deploy model to production?")
85+
deploy_model(trained_model, approved)
86+
```
87+
88+
### Default Response Keywords
89+
90+
By default, alerters recognize these response options:
91+
92+
**Approval:** `approve`, `LGTM`, `ok`, `yes`
93+
**Disapproval:** `decline`, `disapprove`, `no`, `reject`
94+
95+
### Customizing Response Keywords
96+
97+
You can customize the approval and disapproval keywords using alerter parameters:
98+
99+
```python
100+
from zenml.integrations.slack.steps.slack_alerter_ask_step import slack_alerter_ask_step
101+
from zenml.integrations.slack.alerters.slack_alerter import SlackAlerterParameters
102+
103+
# Use custom approval/disapproval keywords
104+
params = SlackAlerterParameters(
105+
approve_msg_options=["deploy", "ship it", "✅"],
106+
disapprove_msg_options=["stop", "cancel", "❌"]
107+
)
108+
109+
approved = slack_alerter_ask_step(
110+
"Deploy model to production?",
111+
params=params
112+
)
113+
```
114+
115+
### Important Notes
116+
117+
- **Return Type**: Ask steps return a boolean value - ensure your pipeline logic handles this correctly
118+
- **Keywords**: Response keywords are case-sensitive (except Slack, which converts to lowercase)
119+
- **Timeout**: If no valid response is received within the timeout period, the step returns `False`
120+
- **Permissions**: Ensure your bot has permissions to read messages in the target channel
121+
51122
<!-- For scarf -->
52123
<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>

‎docs/book/component-guide/alerters/custom.md

Lines changed: 73 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,13 @@ The base abstraction for alerters is very basic, as it only defines two abstract
1515
* `post()` takes a string, posts it to the desired chat service, and returns `True` if the operation succeeded, else `False`.
1616
* `ask()` does the same as `post()`, but after sending the message, it waits until someone approves or rejects the operation from within the chat service (e.g., by sending "approve" / "reject" to the bot as a response). `ask()` then only returns `True` if the operation succeeded and was approved, else `False`.
1717

18+
The `ask()` method is particularly useful for implementing human-in-the-loop workflows. When implementing this method, you should:
19+
- Wait for user responses containing approval keywords (like `"approve"`, `"yes"`, `"ok"`, `"LGTM"`)
20+
- Wait for user responses containing disapproval keywords (like `"reject"`, `"no"`, `"cancel"`, `"stop"`)
21+
- Return `True` only when explicit approval is received
22+
- Return `False` for disapproval, timeout, or any errors
23+
- Consider implementing configurable approval/disapproval keywords via parameters
24+
1825
Then base abstraction looks something like this:
1926

2027
```python
@@ -40,7 +47,7 @@ This is a slimmed-down version of the base implementation. To see the full docst
4047

4148
### Building your own custom alerter
4249

43-
Creating your own custom alerter can be done in three steps:
50+
Creating your own custom alerter can be done in four steps:
4451

4552
1. Create a class that inherits from the `BaseAlerter` and implement the `post()` and `ask()` methods.
4653

@@ -54,18 +61,58 @@ Creating your own custom alerter can be done in three steps:
5461
"""My alerter class."""
5562

5663
def post(
57-
self, message: str, config: Optional[BaseAlerterStepParameters]
64+
self, message: str, params: Optional[BaseAlerterStepParameters]
5865
) -> bool:
5966
"""Post a message to a chat service."""
60-
...
61-
return "Hey, I implemented an alerter."
67+
try:
68+
# Implement your chat service posting logic here
69+
# e.g., send HTTP request to chat API
70+
logging.info(f"Posting message: {message}")
71+
return True
72+
except Exception as e:
73+
logging.error(f"Failed to post message: {e}")
74+
return False
6275

6376
def ask(
64-
self, question: str, config: Optional[BaseAlerterStepParameters]
77+
self, question: str, params: Optional[BaseAlerterStepParameters]
6578
) -> bool:
6679
"""Post a message to a chat service and wait for approval."""
67-
...
68-
return True
80+
try:
81+
# First, post the question
82+
if not self.post(question, params):
83+
return False
84+
85+
# Define default approval/disapproval options
86+
approve_options = ["approve", "yes", "ok", "LGTM"]
87+
disapprove_options = ["reject", "no", "cancel", "stop"]
88+
89+
# Check if custom options are provided in params
90+
if params and hasattr(params, 'approve_msg_options'):
91+
approve_options = params.approve_msg_options
92+
if params and hasattr(params, 'disapprove_msg_options'):
93+
disapprove_options = params.disapprove_msg_options
94+
95+
# Wait for response (implement your chat service polling logic)
96+
# This is a simplified example - you'd implement actual polling
97+
response = self._wait_for_user_response()
98+
99+
if response.lower() in [opt.lower() for opt in approve_options]:
100+
return True
101+
elif response.lower() in [opt.lower() for opt in disapprove_options]:
102+
return False
103+
else:
104+
# Invalid response or timeout
105+
return False
106+
107+
except Exception as e:
108+
print(f"Failed to get approval: {e}")
109+
return False
110+
111+
def _wait_for_user_response(self) -> str:
112+
"""Wait for user response - implement based on your chat service."""
113+
# This is where you'd implement the actual waiting logic
114+
# e.g., polling your chat service API for new messages
115+
return "approve" # Placeholder
69116
```
70117
2. If you need to configure your custom alerter, you can also implement a config object.
71118

@@ -76,7 +123,25 @@ Creating your own custom alerter can be done in three steps:
76123
class MyAlerterConfig(BaseAlerterConfig):
77124
my_param: str
78125
```
79-
3. Finally, you can bring the implementation and the configuration together in a new flavor object.
126+
127+
3. Optionally, you can create custom parameter classes to support configurable approval/disapproval keywords:
128+
129+
```python
130+
from typing import List, Optional
131+
from zenml.alerter.base_alerter import BaseAlerterStepParameters
132+
133+
134+
class MyAlerterParameters(BaseAlerterStepParameters):
135+
"""Custom parameters for MyAlerter."""
136+
137+
# Custom approval/disapproval message options
138+
approve_msg_options: Optional[List[str]] = None
139+
disapprove_msg_options: Optional[List[str]] = None
140+
141+
# Any other custom parameters for your alerter
142+
custom_channel: Optional[str] = None
143+
```
144+
4. Finally, you can bring the implementation and the configuration together in a new flavor object.
80145

81146
```python
82147
from typing import Type, TYPE_CHECKING

‎docs/book/component-guide/alerters/discord.md

Lines changed: 64 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,23 @@ zenml alerter register discord_alerter \
5151
--default_discord_channel_id=<DISCORD_CHANNEL_ID>
5252
```
5353

54+
{% hint style="info" %}
55+
**Using Secrets for Token Management**: Instead of passing your Discord token directly, it's recommended to store it as a ZenML secret and reference it in your alerter configuration. This approach keeps sensitive information secure:
56+
57+
```shell
58+
# Create a secret for your Discord token
59+
zenml secret create discord_secret --discord_token=<DISCORD_TOKEN>
60+
61+
# Register the alerter referencing the secret
62+
zenml alerter register discord_alerter \
63+
--flavor=discord \
64+
--discord_token={{discord_secret.discord_token}} \
65+
--default_discord_channel_id=<DISCORD_CHANNEL_ID>
66+
```
67+
68+
Learn more about [referencing secrets in stack component attributes and settings](https://docs.zenml.io/concepts/secrets#reference-secrets-in-stack-component-attributes-and-settings).
69+
{% endhint %}
70+
5471
After you have registered the `discord_alerter`, you can add it to your stack like this:
5572

5673
```shell
@@ -97,18 +114,64 @@ def my_formatter_step(artifact_to_be_communicated) -> str:
97114
return f"Here is my artifact {artifact_to_be_communicated}!"
98115

99116

117+
@step
118+
def process_approval_response(artifact, approved: bool) -> None:
119+
if approved:
120+
# Proceed with the operation
121+
print(f"User approved! Processing {artifact}")
122+
# Your logic here
123+
else:
124+
print("User disapproved. Skipping operation.")
125+
126+
100127
@pipeline
101128
def my_pipeline(...):
102129
...
103130
artifact_to_be_communicated = ...
104131
message = my_formatter_step(artifact_to_be_communicated)
105132
approved = discord_alerter_ask_step(message)
106-
... # Potentially have different behavior in subsequent steps if `approved`
133+
process_approval_response(artifact_to_be_communicated, approved)
107134

108135
if __name__ == "__main__":
109136
my_pipeline()
110137
```
111138

139+
## Using Custom Approval Keywords
140+
141+
You can customize which words trigger approval or disapproval by using `DiscordAlerterParameters`:
142+
143+
```python
144+
from zenml.integrations.discord.steps.discord_alerter_ask_step import discord_alerter_ask_step
145+
from zenml.integrations.discord.alerters.discord_alerter import DiscordAlerterParameters
146+
147+
# Custom approval/disapproval keywords
148+
params = DiscordAlerterParameters(
149+
approve_msg_options=["deploy", "ship it", "✅"],
150+
disapprove_msg_options=["stop", "cancel", "❌"]
151+
)
152+
153+
approved = discord_alerter_ask_step(
154+
"Deploy model to production?",
155+
params=params
156+
)
157+
```
158+
159+
### Default Response Keywords
160+
161+
By default, the Discord alerter recognizes these keywords:
162+
163+
**Approval:** `approve`, `LGTM`, `ok`, `yes`
164+
**Disapproval:** `decline`, `disapprove`, `no`, `reject`
165+
166+
**Important Notes:**
167+
- The ask step returns a boolean (`True` for approval, `False` for disapproval/timeout)
168+
- **Keywords are case-sensitive** - you must respond with exact case (e.g., `LGTM` not `lgtm`)
169+
- If no valid response is received, the step returns `False`
170+
171+
{% hint style="warning" %}
172+
**Discord Case Sensitivity**: The Discord alerter implementation requires exact case matching for approval keywords. Make sure to respond with the exact case specified (e.g., `LGTM`, not `lgtm`).
173+
{% endhint %}
174+
112175
For more information and a full list of configurable attributes of the Discord alerter, check out the [SDK Docs](https://sdkdocs.zenml.io/latest/integration_code_docs/integrations-discord.html#zenml.integrations.discord) .
113176

114177
<figure><img src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" alt="ZenML Scarf"><figcaption></figcaption></figure>

‎docs/book/component-guide/alerters/slack.md

Lines changed: 43 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,12 @@ zenml alerter register slack_alerter \
5555
--slack_channel_id=<SLACK_CHANNEL_ID>
5656
```
5757

58+
{% hint style="info" %}
59+
**Using Secrets for Token Management**: The example above demonstrates the recommended approach of storing your Slack token as a ZenML secret and referencing it using the `{{secret_name.key}}` syntax. This keeps sensitive information secure and follows security best practices.
60+
61+
Learn more about [referencing secrets in stack component attributes and settings](https://docs.zenml.io/concepts/secrets#reference-secrets-in-stack-component-attributes-and-settings).
62+
{% endhint %}
63+
5864
Here is where you can find the required parameters:
5965

6066
* `<SLACK_CHANNEL_ID>`: The channel ID can be found in the channel details.
@@ -171,7 +177,7 @@ def post_statement() -> None:
171177
)
172178

173179

174-
# Formatting with blocks
180+
# Formatting with blocks and custom approval options
175181
@step
176182
def ask_question() -> bool:
177183
message = ":tada: Should I continue? (Y/N)"
@@ -193,11 +199,20 @@ def ask_question() -> bool:
193199
)
194200
return Client().active_stack.alerter.ask(question=message, params=params)
195201

202+
@step
203+
def process_approval_response(approved: bool) -> None:
204+
if approved:
205+
print("User approved! Continuing with operation...")
206+
# Your logic here
207+
else:
208+
print("User declined. Stopping operation.")
209+
196210

197211
@pipeline(enable_cache=False)
198212
def my_pipeline():
199213
post_statement()
200-
ask_question()
214+
approved = ask_question()
215+
process_approval_response(approved)
201216

202217

203218
if __name__ == "__main__":
@@ -211,25 +226,49 @@ If you want to only use it in a simple manner, you can also use the steps
211226
the Slack integration of ZenML:
212227

213228
```python
214-
from zenml import pipeline
229+
from zenml import pipeline, step
215230
from zenml.integrations.slack.steps.slack_alerter_post_step import (
216231
slack_alerter_post_step
217232
)
218233
from zenml.integrations.slack.steps.slack_alerter_ask_step import (
219234
slack_alerter_ask_step,
220235
)
221236

237+
@step
238+
def process_approval_response(approved: bool) -> None:
239+
if approved:
240+
print("Operation approved!")
241+
else:
242+
print("Operation declined.")
222243

223244
@pipeline(enable_cache=False)
224245
def my_pipeline():
225246
slack_alerter_post_step("Posting a statement.")
226-
slack_alerter_ask_step("Asking a question. Should I continue?")
247+
approved = slack_alerter_ask_step("Asking a question. Should I continue?")
248+
process_approval_response(approved)
227249

228250

229251
if __name__ == "__main__":
230252
my_pipeline()
231253
```
232254

255+
## Default Response Keywords and Ask Step Behavior
256+
257+
The `ask()` method and `slack_alerter_ask_step` recognize these keywords by default:
258+
259+
**Approval:** `approve`, `LGTM`, `ok`, `yes`
260+
**Disapproval:** `decline`, `disapprove`, `no`, `reject`
261+
262+
**Important Notes:**
263+
- The ask step returns a boolean (`True` for approval, `False` for disapproval/timeout)
264+
- **Response keywords are case-insensitive** - keywords are converted to lowercase before matching (e.g., both `LGTM` and `lgtm` work)
265+
- If no valid response is received within the timeout period, the step returns `False`
266+
- The default timeout is 300 seconds (5 minutes) but can be configured
267+
268+
{% hint style="info" %}
269+
**Slack Case Handling**: The Slack alerter implementation automatically converts all response keywords to lowercase before matching, making responses case-insensitive. You can respond with `LGTM`, `lgtm`, or `Lgtm` - they'll all work.
270+
{% endhint %}
271+
233272
For more information and a full list of configurable attributes of the Slack
234273
alerter, check out the [SDK Docs](https://sdkdocs.zenml.io/latest/integration_code_docs/integrations-slack.html#zenml.integrations.slack) .
235274

0 commit comments

Comments
 (0)