Skip to content

Refactoring, speed improvements, optional dependencies #146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
May 23, 2025

Conversation

knopki
Copy link
Member

@knopki knopki commented May 22, 2025

This PR can be characterized as refactoring.

Most notable changes:

  • All cloud connectors are now optional. They are only loaded if they are configured in the configuration file AND dependencies are installed. This increases the cold start time of all utilities. It also speeds up the loading of programs that use the yascheduler client. Cloud connections now need to be set separately, for example as yascheduler[hetzner], but the current settings shouldn't break. See 1128c39 fix Lazy load or separate sub-packages for cloud providers #112
  • Lazier loading, so all utilities (except the main daemon) and the client should start faster. See a91de8f

Also:

  • Removed overly complex code that is of no obvious benefit (some Protocols).
  • Migrate to ruff + pyright instead of big pile of tools
  • Migrate to setuptools
  • Bump hcloud version to 2.x
  • Fix tons of typing errors
  • Migrate all code to Python 3.9 (mostly collections.abc imports and Dict -> dict)

@knopki
Copy link
Member Author

knopki commented May 22, 2025

@blokhin Please, allow jakebailey/pyright-action@* action in the tilde-lab or tilde-lab/yascheduler.

yascheduler.conf Outdated
engines_dir = %(data_dir)s/engines

[clouds]
hetzner_token = ngzcJHBUZ0MdbjrbrNaVOtDXmE8iXIpGvnTYVPx02wgHXsKxcbMCh2uhnGIbHj8X
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@knopki I just discarded the token, no worries

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OMG, thanks

yascheduler.conf Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file should not be committed, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, definitely. It's for testing only.

@@ -50,7 +50,8 @@ def upcloud_create_node_sync(
user_data=cloud_config.render() if cloud_config else None,
)
)
ip_addr = server.get_public_ip()
ip_addr = cast(Optional[str], server.get_public_ip())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow, a modern Python becomes so complex!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just a combination of a shitty library and my "I don't want to deal with UpCloud"

@blokhin blokhin merged commit fe322ff into tilde-lab:master May 23, 2025
4 of 5 checks passed
@knopki knopki deleted the fix/refactoring branch May 23, 2025 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Lazy load or separate sub-packages for cloud providers
2 participants