Skip to content

0.84.2
Compare
Choose a tag to compare
@schustmi schustmi released this 06 Aug 19:18
· 17 commits to main since this release
0.84.2
d626435
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

The 0.84.2 release brings significant performance improvements and reliability enhancements that strengthen ZenML's orchestration capabilities. This release features a major refactor of the Kubernetes orchestrator with enhanced restart capabilities, faster pipeline compilation for large pipelines, critical security fixes, and improved logging performance.

Kubernetes Orchestrator

  • Enhanced Robustness: Complete rework to use Jobs instead of raw pods for orchestrator execution with state reconstruction support, enabling automatic restarts for a much more robust orchestration experience
  • Deprecated Settings: Several settings attributes are now deprecated due to the shift from pods to Jobs, streamlining the configuration interface
  • Updated Logging Behavior: The orchestrator pod no longer streams logs from step pods directly, improving performance and resource usage

🚀 Improvements

  • Faster Pipeline Compilation: Significantly improved pipeline compilation performance for large pipelines

🛠️ Fixes

  • Run Creation Deadlock: Fixed deadlock that could happen if two steps try to create a run at the same time
  • Security Enhancement: Enhanced path materializer to properly validate symlinks and hardlinks to prevent path traversal attacks
  • Logging Performance: Improved logging thread performance by avoiding unnecessary sleep when shutdown is requested
  • WandB Integration: Fixed WandB experiment tracker flavor initialization when the integration is not installed
  • Type Annotations: Fixed type annotations for experiment_tracker and step_operator parameters to accept both string and boolean values

What's Changed

  • Add version 0.84.0 to legacy docs by @github-actions[bot] in #3863
  • Add 0.84.1 to the migration tests by @github-actions[bot] in #3865
  • Fix wandb flavor needing integration to be installed by @stefannica in #3871
  • Verify symlinks and hardlinks in the path materializer by @stefannica in #3870
  • Don't sleep in logging thread if shutdown was requested by @schustmi in #3872
  • Fix run creation deadlock by @schustmi in #3876
  • Fix nonlinear scaling of pipeline compilation time by @schustmi in #3873
  • Fix type annotations by @schustmi in #3878
  • Orchestrator pod restarts by @schustmi in #3869

Full Changelog: 0.84.1...0.84.2

Contributors

stefannica and schustmi
Assets 2
Loading