🧑‍💻 Support import of model classes before connecting to an instance

[ap--]

Hi 👋

This adds support for importing lamindb model classes before running ln.connect(...) via lazy import proxies for all objects that are only available after connect. This plays much nicer with modern Python code style, where all imports are on the top of the file. After the user connects to an instance, instantiating the proxy classes returns instances of the corresponding models and calling classmethods on the proxy delegates to the model class.

Notes

This needs tests, since I would bet that there are some edge cases where it breaks down. My guess is that auto-completion in jupyter might behave differently, but it needs to be checked.

I also noticed that lamindb imports a lot of submodules eagerly once connect happens, and I am not sure why you'd want that. I wonder if this enabled a better experience for users in jupyter notebooks and that's why you did it that way. It would be great to understand the reasoning behind it.

In addition I changed the submodule import behavior in this PR slightly, and made the import of submodules "base", "errors", and "setup" also lazy (the three submodules that are eagerly imported before connect). I also made the module level __getattr__ actually error with the correct Exception in case a user wants to access something that doesn't exists. For example import lamindb as ln; ln.blah will now correctly raise an AttributeError.

I'll continue with this a bit next week.

[ap--]

The failing tests are currently due to my code changes. I'll debug what I am missing.

[ap--]

some notes for next week

It should probably be revisited why importlib.reload is needed at all in lamindb. My guess is some of the issues I am seeing are due to the caveats around https://docs.python.org/3/library/importlib.html#importlib.reload and "from imports".

If it's required and can't be avoided it might be useful to adopt a ruff style for imports that prevents named from imports from within lamindb.

[codecov]

Codecov Report

Attention: Patch coverage is 68.31683% with 32 lines in your changes missing coverage. Please review.

Project coverage is 91.21%. Comparing base (8265549) to head (370fd41).
Report is 31 commits behind head on main.

Files with missing lines	Patch %	Lines
lamindb/_lazy_import.py	52.45%	29 Missing ⚠️
lamindb/__init__.py	92.10%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2825      +/-   ##
==========================================
- Coverage   91.84%   91.21%   -0.63%     
==========================================
  Files          69       66       -3     
  Lines       10807     9315    -1492     
==========================================
- Hits         9926     8497    -1429     
+ Misses        881      818      -63     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

Deployment URL: https://473199ba.lamindb.pages.dev

[ap--]

Closing in favour of #2849