Skip to content

Conversation

marwan37
Copy link
Contributor

@marwan37 marwan37 commented Mar 31, 2025

Describe changes

  • This PR fixes incorrect step access method references throughout the docs.
  • Updated 4 documents to replace the incorrect method with the working approach

I noticed this issue while reviewing the MLFlow documentation. The docs show examples using last_run.get_step("<STEP_NAME>"), but this method doesn't exist on PipelineRunResponse and throws an AttributeError. The correct way to access steps is through the steps dictionary: last_run.steps["<STEP_NAME>"].

Pre-requisites

Please ensure you have done the following:

  • I have read the CONTRIBUTING.md document.
  • I have added tests to cover my changes.
  • I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.
  • IMPORTANT: I made sure that my changes are reflected properly in the following resources:
    • ZenML Docs
    • Dashboard: Needs to be communicated to the frontend team.
    • Templates: Might need adjustments (that are not reflected in the template tests) in case of non-breaking changes and deprecations.
    • Projects: Depending on the version dependencies, different projects might get affected.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Other (add details above)

@marwan37 marwan37 added the documentation Improvements or additions to documentation label Mar 31, 2025
Copy link
Contributor

coderabbitai bot commented Mar 31, 2025

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai plan to trigger planning for file edits and PR creation.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@CLAassistant
Copy link

CLAassistant commented Mar 31, 2025

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

Documentation Link Check Results

Absolute links check passed
Relative links check passed
Last checked: 2025-03-31 18:19:28 UTC

@htahir1
Copy link
Contributor

htahir1 commented Mar 31, 2025

@marwan37 please do sign the CLA as commented above, and we can then merge :-)

@strickvl strickvl added the internal To filter out internal PRs and issues label Mar 31, 2025
@htahir1 htahir1 merged commit b17c00e into develop Apr 1, 2025
14 checks passed
@htahir1 htahir1 deleted the docs/fix-step-acess-method branch April 1, 2025 13:20
htahir1 pushed a commit that referenced this pull request Apr 1, 2025
strickvl added a commit that referenced this pull request Apr 1, 2025
…s/projects) (#3485)

* Add "Testing" section to the Table of Contents in user guide (#3481)

* Updated the Table of Contents to include a new entry for "Testing", linking to the README.md file. This enhances navigation for users seeking information on testing within the documentation.

This change improves the overall structure and accessibility of the user guide.

* Add "Tutorials" section to the Table of Contents and introduce a new guide on managing scheduled pipelines (#3482)

* Updated the Table of Contents to include a new entry for "Tutorials", linking to the managing-scheduled-pipelines.md file. This enhances navigation for users seeking information on scheduling within the documentation.

* Added a comprehensive guide on managing scheduled pipeline executions in ZenML, covering topics such as creating, managing, and troubleshooting schedules across different orchestrators.

This change improves the overall structure and accessibility of the user guide, providing valuable resources for users interested in pipeline scheduling.

* Update "Tutorials" section in Table of Contents and create README for tutorials (#3483)

* Modified the Table of Contents to link the "Tutorials" section to the new README.md file, enhancing user navigation for tutorial resources.
* Introduced a README.md file in the tutorials directory, which will serve as an overview for available tutorials, including a link to the guide on managing scheduled pipelines.

This change improves the documentation structure and accessibility for users seeking tutorial information.

* section instead of dropdown

* Complete section 1

* docs: Complete sections 1 and 2 of scheduled pipelines guide

- Added detailed explanation of how scheduling works in ZenML
- Created comprehensive orchestrator support comparison table
- Added advanced scheduling patterns and code examples
- Added orchestrator-specific implementation details
- Improved schedule naming and management best practices

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* docs: Complete section 3 of scheduled pipelines guide

- Added comprehensive guidance on viewing and monitoring schedules
- Detailed the workflow for updating schedules across orchestrators
- Provided orchestrator-specific schedule management code examples
- Added complete instructions for schedule deletion and cleanup

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* docs: Complete scheduled pipelines guide

- Consolidated sections 3-5 into a more coherent structure
- Completed section 3 with detailed orchestrator-specific management
- Added comprehensive orphaned schedule cleanup implementation
- Added complete best practices section with code examples
- Added alternative scheduling approaches through external systems
- Added troubleshooting guide with common issues and resolutions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Update schedule limitations column

* Refine scheduling limitations in orchestrator comparison table

- Updated the limitations for Airflow, Databricks, HyperAI, Kubernetes, SageMaker, and Vertex AI to clarify requirements for schedule changes and management.
- Added a note regarding ZenML's current inability to update or delete orchestrator schedules directly through its API, emphasising the need for native tools.

This change enhances the clarity and accuracy of the documentation for users managing scheduled pipelines.

* Enhance scheduled pipelines documentation with updated alerter setup

- Revised the section on setting up monitoring for scheduled pipeline runs to include the use of hooks for success and failure notifications.
- Updated code examples to reflect the new approach using `alerter_success_hook` and `alerter_failure_hook`, improving clarity and usability for users implementing monitoring in their pipelines.

This change aims to provide clearer guidance on integrating monitoring within scheduled pipelines, ensuring users can effectively manage alerts.

* Enhance pipeline scheduling guide with improved examples

- Improve limitations table in section 1.2 with technical details for each orchestrator
- Fix alerter hooks example code in sections 3.1 and 4.2
- Expand section 5 with comprehensive alternative scheduling implementations
- Add detailed script examples for using cloud provider services
- Add robust CI/CD scheduling examples with environment handling
- Expand section 6 with in-depth diagnostic tools and scripts
- Add systematic schedule validation approach
- Add orchestrator-specific diagnostic commands

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Fix missing imports in code examples

* Fix pipeline triggering code to use correct ZenML API

* Try alternative version as cookbook

* Enhance scheduled pipelines documentation with troubleshooting guidance

- Updated the troubleshooting section to include practical fixes for common issues encountered with scheduled pipelines.
- Added code examples for verifying cron expressions and checking service account permissions.
- Included a new section on cleaning up orphaned schedules in Vertex AI and finding failing scheduled runs.

This change aims to provide users with clearer guidance and actionable steps to resolve issues related to scheduled pipelines, improving overall usability and effectiveness.

* Add tutorial on organizing pipelines and models

- Introduced a new tutorial titled "Organizing Pipelines and Models" to the user guide, providing a comprehensive step-by-step approach to effectively manage ML assets in ZenML.
- Updated the table of contents to include a link to the new tutorial, enhancing navigation for users seeking guidance on organization strategies.

This addition aims to improve user understanding of asset management within ZenML, facilitating better project organization and collaboration.

* Update scheduled pipelines documentation to reflect tutorial changes

- Renamed the tutorial on managing scheduled pipelines to "Managing Scheduled Pipelines in ZenML: A Cookbook" for clarity and consistency.
- Updated the table of contents to link to the revised tutorial, ensuring users can easily navigate to the new content.
- Removed the outdated tutorial file "managing-scheduled-pipelines-v2.md" to streamline the documentation.

This update aims to enhance user understanding of scheduled pipelines in ZenML by providing a more focused and practical guide, improving overall usability and accessibility of the documentation.

* Update .gitignore to include design directory and CLAUDE.md

- Added the 'design/' directory to the .gitignore file to prevent tracking of design documents.
- Included 'CLAUDE.md' in the .gitignore to exclude it from version control.

This update aims to streamline the repository by ensuring that unnecessary files are not tracked, improving overall project organisation.

* Enhance scheduled pipelines documentation with additional reference information

- Added a hint section to the tutorial on managing scheduled pipelines, directing users to the full reference documentation on schedules.
- Improved the readability of the text by breaking a long sentence into two.

This update aims to provide users with better guidance and resources for understanding and implementing scheduled pipelines in ZenML, enhancing the overall usability of the documentation.

* Add links

* v3

* add central links

* Update user guide tutorial on organizing pipelines and models

- Added a hyperlink to the ZenML Pro account for clarity and ease of access.
- Revised the list of follow-up actions to include "Ways to trigger pipelines" instead of "Creating dashboards and visualizations based on your tags."

This update aims to enhance the user experience by providing direct links and updated content, ensuring users have the most relevant information for managing their ML workflows in ZenML.

* Fix line formatting

* Fix code formatting

* Fix Model import in tagging docs

* Fix tagging docs errors

* Check and test the organization tutorial

* Fix link

* Apply suggestions from code review

Co-authored-by: Hamza Tahir <[email protected]>

* Testing inline note

* Trying different style

* Update managing-scheduled-pipelines tutorial to clarify pipeline scheduling

- Added a note explaining that running the pipeline creates a schedule in the ZenML metadata store and a scheduled run in the orchestrator.
- Emphasised the importance of using descriptive schedule names for better organisation.

This update aims to improve user understanding of scheduled pipelines and best practices in ZenML.

* Enhance managing-scheduled-pipelines tutorial with GCP verification steps

- Added a new section to verify the creation of schedules in Vertex AI using the Google Cloud SDK.
- Included example code for listing Vertex schedules and checking their details.
- Added a warning hint to remind users to replace the region with their actual Vertex AI region.

This update aims to provide users with a clearer understanding of how to confirm their scheduled pipelines in GCP.

* Add CLI output

* Enhance managing-scheduled-pipelines tutorial with additional instructions

- Added a note to rerun the pipeline after creating a new schedule, including a command example for clarity.
- This update aims to improve user guidance on managing scheduled pipelines effectively.

* Update managing-scheduled-pipelines tutorial to include a hyperlink for alerter registration

- Enhanced the note regarding alerter registration by adding a hyperlink to the relevant documentation.
- This update aims to provide users with direct access to information on registering alerters, improving the overall clarity and usability of the tutorial.

* Use an enum

* Add instructions for deleting schedules on GCP in managing-scheduled-pipelines tutorial

- Introduced a new section detailing the process for deleting schedules from Vertex AI using the Google Cloud SDK.
- Included example code for listing Vertex schedules and deleting a specific schedule based on its display name.
- This update aims to enhance user guidance on managing scheduled pipelines effectively in GCP.

* add missing section back!

* Add timezone section

- Clarified the behaviour of ZenML regarding timezone-aware and naive datetime inputs for scheduled runs.
- Added detailed examples demonstrating how to set schedules using local and UTC timezones.
- Included reminders about the interpretation of cron expressions in relation to system timezones.

This update aims to help users avoid confusion with scheduled pipeline timings, ensuring they understand how to manage timezones effectively.

* Add links in the next steps sections

* Add screenshot

* Enhance scheduled pipelines tutorial with clarified cleanup instructions

Improve the managed-scheduled-pipelines tutorial with:
- Added comprehensive section on timezone handling to help users avoid timezone confusion
- Restructured the cleanup sections to clearly emphasize the need for dual deletion (both ZenML and orchestrator)
- Added warning hints and verification steps to ensure users properly clean up orchestrator schedules
- Improved section on updating schedules with cross-references between sections
- Added more details on Vertex AI schedule management specifically

* Improve structure consistency in scheduled pipelines tutorial

- Reorganize the sections with proper sub-sections to maintain consistent structure
- Fixed inconsistency with Step 3.1, renaming it to Step 3.2 and adding a proper Step 3.1
- Added proper subsections to Step 4 and improved descriptions for each part of the process
- Enhanced explanations in Step 6

---------

Co-authored-by: Claude <[email protected]>
Co-authored-by: Hamza Tahir <[email protected]>
strickvl added a commit that referenced this pull request Apr 1, 2025
…s/projects) (#3485)

* Add "Testing" section to the Table of Contents in user guide (#3481)

* Updated the Table of Contents to include a new entry for "Testing", linking to the README.md file. This enhances navigation for users seeking information on testing within the documentation.

This change improves the overall structure and accessibility of the user guide.

* Add "Tutorials" section to the Table of Contents and introduce a new guide on managing scheduled pipelines (#3482)

* Updated the Table of Contents to include a new entry for "Tutorials", linking to the managing-scheduled-pipelines.md file. This enhances navigation for users seeking information on scheduling within the documentation.

* Added a comprehensive guide on managing scheduled pipeline executions in ZenML, covering topics such as creating, managing, and troubleshooting schedules across different orchestrators.

This change improves the overall structure and accessibility of the user guide, providing valuable resources for users interested in pipeline scheduling.

* Update "Tutorials" section in Table of Contents and create README for tutorials (#3483)

* Modified the Table of Contents to link the "Tutorials" section to the new README.md file, enhancing user navigation for tutorial resources.
* Introduced a README.md file in the tutorials directory, which will serve as an overview for available tutorials, including a link to the guide on managing scheduled pipelines.

This change improves the documentation structure and accessibility for users seeking tutorial information.

* section instead of dropdown

* Complete section 1

* docs: Complete sections 1 and 2 of scheduled pipelines guide

- Added detailed explanation of how scheduling works in ZenML
- Created comprehensive orchestrator support comparison table
- Added advanced scheduling patterns and code examples
- Added orchestrator-specific implementation details
- Improved schedule naming and management best practices

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* docs: Complete section 3 of scheduled pipelines guide

- Added comprehensive guidance on viewing and monitoring schedules
- Detailed the workflow for updating schedules across orchestrators
- Provided orchestrator-specific schedule management code examples
- Added complete instructions for schedule deletion and cleanup

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* docs: Complete scheduled pipelines guide

- Consolidated sections 3-5 into a more coherent structure
- Completed section 3 with detailed orchestrator-specific management
- Added comprehensive orphaned schedule cleanup implementation
- Added complete best practices section with code examples
- Added alternative scheduling approaches through external systems
- Added troubleshooting guide with common issues and resolutions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Update schedule limitations column

* Refine scheduling limitations in orchestrator comparison table

- Updated the limitations for Airflow, Databricks, HyperAI, Kubernetes, SageMaker, and Vertex AI to clarify requirements for schedule changes and management.
- Added a note regarding ZenML's current inability to update or delete orchestrator schedules directly through its API, emphasising the need for native tools.

This change enhances the clarity and accuracy of the documentation for users managing scheduled pipelines.

* Enhance scheduled pipelines documentation with updated alerter setup

- Revised the section on setting up monitoring for scheduled pipeline runs to include the use of hooks for success and failure notifications.
- Updated code examples to reflect the new approach using `alerter_success_hook` and `alerter_failure_hook`, improving clarity and usability for users implementing monitoring in their pipelines.

This change aims to provide clearer guidance on integrating monitoring within scheduled pipelines, ensuring users can effectively manage alerts.

* Enhance pipeline scheduling guide with improved examples

- Improve limitations table in section 1.2 with technical details for each orchestrator
- Fix alerter hooks example code in sections 3.1 and 4.2
- Expand section 5 with comprehensive alternative scheduling implementations
- Add detailed script examples for using cloud provider services
- Add robust CI/CD scheduling examples with environment handling
- Expand section 6 with in-depth diagnostic tools and scripts
- Add systematic schedule validation approach
- Add orchestrator-specific diagnostic commands

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Fix missing imports in code examples

* Fix pipeline triggering code to use correct ZenML API

* Try alternative version as cookbook

* Enhance scheduled pipelines documentation with troubleshooting guidance

- Updated the troubleshooting section to include practical fixes for common issues encountered with scheduled pipelines.
- Added code examples for verifying cron expressions and checking service account permissions.
- Included a new section on cleaning up orphaned schedules in Vertex AI and finding failing scheduled runs.

This change aims to provide users with clearer guidance and actionable steps to resolve issues related to scheduled pipelines, improving overall usability and effectiveness.

* Add tutorial on organizing pipelines and models

- Introduced a new tutorial titled "Organizing Pipelines and Models" to the user guide, providing a comprehensive step-by-step approach to effectively manage ML assets in ZenML.
- Updated the table of contents to include a link to the new tutorial, enhancing navigation for users seeking guidance on organization strategies.

This addition aims to improve user understanding of asset management within ZenML, facilitating better project organization and collaboration.

* Update scheduled pipelines documentation to reflect tutorial changes

- Renamed the tutorial on managing scheduled pipelines to "Managing Scheduled Pipelines in ZenML: A Cookbook" for clarity and consistency.
- Updated the table of contents to link to the revised tutorial, ensuring users can easily navigate to the new content.
- Removed the outdated tutorial file "managing-scheduled-pipelines-v2.md" to streamline the documentation.

This update aims to enhance user understanding of scheduled pipelines in ZenML by providing a more focused and practical guide, improving overall usability and accessibility of the documentation.

* Update .gitignore to include design directory and CLAUDE.md

- Added the 'design/' directory to the .gitignore file to prevent tracking of design documents.
- Included 'CLAUDE.md' in the .gitignore to exclude it from version control.

This update aims to streamline the repository by ensuring that unnecessary files are not tracked, improving overall project organisation.

* Enhance scheduled pipelines documentation with additional reference information

- Added a hint section to the tutorial on managing scheduled pipelines, directing users to the full reference documentation on schedules.
- Improved the readability of the text by breaking a long sentence into two.

This update aims to provide users with better guidance and resources for understanding and implementing scheduled pipelines in ZenML, enhancing the overall usability of the documentation.

* Add links

* v3

* add central links

* Update user guide tutorial on organizing pipelines and models

- Added a hyperlink to the ZenML Pro account for clarity and ease of access.
- Revised the list of follow-up actions to include "Ways to trigger pipelines" instead of "Creating dashboards and visualizations based on your tags."

This update aims to enhance the user experience by providing direct links and updated content, ensuring users have the most relevant information for managing their ML workflows in ZenML.

* Fix line formatting

* Fix code formatting

* Fix Model import in tagging docs

* Fix tagging docs errors

* Check and test the organization tutorial

* Fix link

* Apply suggestions from code review

Co-authored-by: Hamza Tahir <[email protected]>

* Testing inline note

* Trying different style

* Update managing-scheduled-pipelines tutorial to clarify pipeline scheduling

- Added a note explaining that running the pipeline creates a schedule in the ZenML metadata store and a scheduled run in the orchestrator.
- Emphasised the importance of using descriptive schedule names for better organisation.

This update aims to improve user understanding of scheduled pipelines and best practices in ZenML.

* Enhance managing-scheduled-pipelines tutorial with GCP verification steps

- Added a new section to verify the creation of schedules in Vertex AI using the Google Cloud SDK.
- Included example code for listing Vertex schedules and checking their details.
- Added a warning hint to remind users to replace the region with their actual Vertex AI region.

This update aims to provide users with a clearer understanding of how to confirm their scheduled pipelines in GCP.

* Add CLI output

* Enhance managing-scheduled-pipelines tutorial with additional instructions

- Added a note to rerun the pipeline after creating a new schedule, including a command example for clarity.
- This update aims to improve user guidance on managing scheduled pipelines effectively.

* Update managing-scheduled-pipelines tutorial to include a hyperlink for alerter registration

- Enhanced the note regarding alerter registration by adding a hyperlink to the relevant documentation.
- This update aims to provide users with direct access to information on registering alerters, improving the overall clarity and usability of the tutorial.

* Use an enum

* Add instructions for deleting schedules on GCP in managing-scheduled-pipelines tutorial

- Introduced a new section detailing the process for deleting schedules from Vertex AI using the Google Cloud SDK.
- Included example code for listing Vertex schedules and deleting a specific schedule based on its display name.
- This update aims to enhance user guidance on managing scheduled pipelines effectively in GCP.

* add missing section back!

* Add timezone section

- Clarified the behaviour of ZenML regarding timezone-aware and naive datetime inputs for scheduled runs.
- Added detailed examples demonstrating how to set schedules using local and UTC timezones.
- Included reminders about the interpretation of cron expressions in relation to system timezones.

This update aims to help users avoid confusion with scheduled pipeline timings, ensuring they understand how to manage timezones effectively.

* Add links in the next steps sections

* Add screenshot

* Enhance scheduled pipelines tutorial with clarified cleanup instructions

Improve the managed-scheduled-pipelines tutorial with:
- Added comprehensive section on timezone handling to help users avoid timezone confusion
- Restructured the cleanup sections to clearly emphasize the need for dual deletion (both ZenML and orchestrator)
- Added warning hints and verification steps to ensure users properly clean up orchestrator schedules
- Improved section on updating schedules with cross-references between sections
- Added more details on Vertex AI schedule management specifically

* Improve structure consistency in scheduled pipelines tutorial

- Reorganize the sections with proper sub-sections to maintain consistent structure
- Fixed inconsistency with Step 3.1, renaming it to Step 3.2 and adding a proper Step 3.1
- Added proper subsections to Step 4 and improved descriptions for each part of the process
- Enhanced explanations in Step 6

---------

Co-authored-by: Claude <[email protected]>
Co-authored-by: Hamza Tahir <[email protected]>
(cherry picked from commit d2803ab)
@strickvl strickvl mentioned this pull request Apr 1, 2025
12 tasks
strickvl added a commit that referenced this pull request Apr 1, 2025
…s/projects) (#3485) (#3487)

* Add "Testing" section to the Table of Contents in user guide (#3481)

* Updated the Table of Contents to include a new entry for "Testing", linking to the README.md file. This enhances navigation for users seeking information on testing within the documentation.

This change improves the overall structure and accessibility of the user guide.

* Add "Tutorials" section to the Table of Contents and introduce a new guide on managing scheduled pipelines (#3482)

* Updated the Table of Contents to include a new entry for "Tutorials", linking to the managing-scheduled-pipelines.md file. This enhances navigation for users seeking information on scheduling within the documentation.

* Added a comprehensive guide on managing scheduled pipeline executions in ZenML, covering topics such as creating, managing, and troubleshooting schedules across different orchestrators.

This change improves the overall structure and accessibility of the user guide, providing valuable resources for users interested in pipeline scheduling.

* Update "Tutorials" section in Table of Contents and create README for tutorials (#3483)

* Modified the Table of Contents to link the "Tutorials" section to the new README.md file, enhancing user navigation for tutorial resources.
* Introduced a README.md file in the tutorials directory, which will serve as an overview for available tutorials, including a link to the guide on managing scheduled pipelines.

This change improves the documentation structure and accessibility for users seeking tutorial information.

* section instead of dropdown

* Complete section 1

* docs: Complete sections 1 and 2 of scheduled pipelines guide

- Added detailed explanation of how scheduling works in ZenML
- Created comprehensive orchestrator support comparison table
- Added advanced scheduling patterns and code examples
- Added orchestrator-specific implementation details
- Improved schedule naming and management best practices

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* docs: Complete section 3 of scheduled pipelines guide

- Added comprehensive guidance on viewing and monitoring schedules
- Detailed the workflow for updating schedules across orchestrators
- Provided orchestrator-specific schedule management code examples
- Added complete instructions for schedule deletion and cleanup

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* docs: Complete scheduled pipelines guide

- Consolidated sections 3-5 into a more coherent structure
- Completed section 3 with detailed orchestrator-specific management
- Added comprehensive orphaned schedule cleanup implementation
- Added complete best practices section with code examples
- Added alternative scheduling approaches through external systems
- Added troubleshooting guide with common issues and resolutions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Update schedule limitations column

* Refine scheduling limitations in orchestrator comparison table

- Updated the limitations for Airflow, Databricks, HyperAI, Kubernetes, SageMaker, and Vertex AI to clarify requirements for schedule changes and management.
- Added a note regarding ZenML's current inability to update or delete orchestrator schedules directly through its API, emphasising the need for native tools.

This change enhances the clarity and accuracy of the documentation for users managing scheduled pipelines.

* Enhance scheduled pipelines documentation with updated alerter setup

- Revised the section on setting up monitoring for scheduled pipeline runs to include the use of hooks for success and failure notifications.
- Updated code examples to reflect the new approach using `alerter_success_hook` and `alerter_failure_hook`, improving clarity and usability for users implementing monitoring in their pipelines.

This change aims to provide clearer guidance on integrating monitoring within scheduled pipelines, ensuring users can effectively manage alerts.

* Enhance pipeline scheduling guide with improved examples

- Improve limitations table in section 1.2 with technical details for each orchestrator
- Fix alerter hooks example code in sections 3.1 and 4.2
- Expand section 5 with comprehensive alternative scheduling implementations
- Add detailed script examples for using cloud provider services
- Add robust CI/CD scheduling examples with environment handling
- Expand section 6 with in-depth diagnostic tools and scripts
- Add systematic schedule validation approach
- Add orchestrator-specific diagnostic commands

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

* Fix missing imports in code examples

* Fix pipeline triggering code to use correct ZenML API

* Try alternative version as cookbook

* Enhance scheduled pipelines documentation with troubleshooting guidance

- Updated the troubleshooting section to include practical fixes for common issues encountered with scheduled pipelines.
- Added code examples for verifying cron expressions and checking service account permissions.
- Included a new section on cleaning up orphaned schedules in Vertex AI and finding failing scheduled runs.

This change aims to provide users with clearer guidance and actionable steps to resolve issues related to scheduled pipelines, improving overall usability and effectiveness.

* Add tutorial on organizing pipelines and models

- Introduced a new tutorial titled "Organizing Pipelines and Models" to the user guide, providing a comprehensive step-by-step approach to effectively manage ML assets in ZenML.
- Updated the table of contents to include a link to the new tutorial, enhancing navigation for users seeking guidance on organization strategies.

This addition aims to improve user understanding of asset management within ZenML, facilitating better project organization and collaboration.

* Update scheduled pipelines documentation to reflect tutorial changes

- Renamed the tutorial on managing scheduled pipelines to "Managing Scheduled Pipelines in ZenML: A Cookbook" for clarity and consistency.
- Updated the table of contents to link to the revised tutorial, ensuring users can easily navigate to the new content.
- Removed the outdated tutorial file "managing-scheduled-pipelines-v2.md" to streamline the documentation.

This update aims to enhance user understanding of scheduled pipelines in ZenML by providing a more focused and practical guide, improving overall usability and accessibility of the documentation.

* Update .gitignore to include design directory and CLAUDE.md

- Added the 'design/' directory to the .gitignore file to prevent tracking of design documents.
- Included 'CLAUDE.md' in the .gitignore to exclude it from version control.

This update aims to streamline the repository by ensuring that unnecessary files are not tracked, improving overall project organisation.

* Enhance scheduled pipelines documentation with additional reference information

- Added a hint section to the tutorial on managing scheduled pipelines, directing users to the full reference documentation on schedules.
- Improved the readability of the text by breaking a long sentence into two.

This update aims to provide users with better guidance and resources for understanding and implementing scheduled pipelines in ZenML, enhancing the overall usability of the documentation.

* Add links

* v3

* add central links

* Update user guide tutorial on organizing pipelines and models

- Added a hyperlink to the ZenML Pro account for clarity and ease of access.
- Revised the list of follow-up actions to include "Ways to trigger pipelines" instead of "Creating dashboards and visualizations based on your tags."

This update aims to enhance the user experience by providing direct links and updated content, ensuring users have the most relevant information for managing their ML workflows in ZenML.

* Fix line formatting

* Fix code formatting

* Fix Model import in tagging docs

* Fix tagging docs errors

* Check and test the organization tutorial

* Fix link

* Apply suggestions from code review

Co-authored-by: Hamza Tahir <[email protected]>

* Testing inline note

* Trying different style

* Update managing-scheduled-pipelines tutorial to clarify pipeline scheduling

- Added a note explaining that running the pipeline creates a schedule in the ZenML metadata store and a scheduled run in the orchestrator.
- Emphasised the importance of using descriptive schedule names for better organisation.

This update aims to improve user understanding of scheduled pipelines and best practices in ZenML.

* Enhance managing-scheduled-pipelines tutorial with GCP verification steps

- Added a new section to verify the creation of schedules in Vertex AI using the Google Cloud SDK.
- Included example code for listing Vertex schedules and checking their details.
- Added a warning hint to remind users to replace the region with their actual Vertex AI region.

This update aims to provide users with a clearer understanding of how to confirm their scheduled pipelines in GCP.

* Add CLI output

* Enhance managing-scheduled-pipelines tutorial with additional instructions

- Added a note to rerun the pipeline after creating a new schedule, including a command example for clarity.
- This update aims to improve user guidance on managing scheduled pipelines effectively.

* Update managing-scheduled-pipelines tutorial to include a hyperlink for alerter registration

- Enhanced the note regarding alerter registration by adding a hyperlink to the relevant documentation.
- This update aims to provide users with direct access to information on registering alerters, improving the overall clarity and usability of the tutorial.

* Use an enum

* Add instructions for deleting schedules on GCP in managing-scheduled-pipelines tutorial

- Introduced a new section detailing the process for deleting schedules from Vertex AI using the Google Cloud SDK.
- Included example code for listing Vertex schedules and deleting a specific schedule based on its display name.
- This update aims to enhance user guidance on managing scheduled pipelines effectively in GCP.

* add missing section back!

* Add timezone section

- Clarified the behaviour of ZenML regarding timezone-aware and naive datetime inputs for scheduled runs.
- Added detailed examples demonstrating how to set schedules using local and UTC timezones.
- Included reminders about the interpretation of cron expressions in relation to system timezones.

This update aims to help users avoid confusion with scheduled pipeline timings, ensuring they understand how to manage timezones effectively.

* Add links in the next steps sections

* Add screenshot

* Enhance scheduled pipelines tutorial with clarified cleanup instructions

Improve the managed-scheduled-pipelines tutorial with:
- Added comprehensive section on timezone handling to help users avoid timezone confusion
- Restructured the cleanup sections to clearly emphasize the need for dual deletion (both ZenML and orchestrator)
- Added warning hints and verification steps to ensure users properly clean up orchestrator schedules
- Improved section on updating schedules with cross-references between sections
- Added more details on Vertex AI schedule management specifically

* Improve structure consistency in scheduled pipelines tutorial

- Reorganize the sections with proper sub-sections to maintain consistent structure
- Fixed inconsistency with Step 3.1, renaming it to Step 3.2 and adding a proper Step 3.1
- Added proper subsections to Step 4 and improved descriptions for each part of the process
- Enhanced explanations in Step 6

---------

Co-authored-by: Claude <[email protected]>
Co-authored-by: Hamza Tahir <[email protected]>
(cherry picked from commit d2803ab)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation internal To filter out internal PRs and issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants