Skip to content

Update ABSStore for compatibility with newer azure.storage.blob. #759

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Jun 1, 2021

Conversation

TomAugspurger
Copy link
Contributor

@TomAugspurger TomAugspurger commented May 25, 2021

This continues #620 to update the Azure ABSStore to be compatible with newer versions of azure.storage.blob.

In addition to @jhamman's compatibility changes, I made a few changes to make this easier to test. The previous API had users pass various pieces of information like the container name, storage account name, account key, etc. Now we just accept an instance of a BlobServiceClient. I've deprecated the old way.

>>> ABSStore("my_container", account_name="account_name", account_key="account_key")  # old
>>> ABSStore(client=azure.storage.blob.ContainerClient(...))  # new

I figure the less we care about the details of azure.storage.blob the better.

I haven't attempted a compatibility layer with older versions of azure.storage.blob. IMO, it's not worth the effort. I think the common case of users passing a storage account name, a container name, and an account key will continue to work fine (with a warning).

Closes #618.

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/tutorial.rst
  • Changes documented in docs/release.rst
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

@pep8speaks
Copy link

pep8speaks commented May 25, 2021

Hello @TomAugspurger! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-06-01 19:14:09 UTC

Tom Augspurger added 2 commits May 24, 2021 20:42
@TomAugspurger
Copy link
Contributor Author

cc @tjcrone.

@TomAugspurger TomAugspurger changed the title Fix/absstore Update ABSStore for compatibility with newer azure.storage.blob. May 25, 2021
@jakirkham
Copy link
Member

cc @shikharsg @jhamman

@jakirkham
Copy link
Member

Welcome Tom 😄 Thanks for working on this!

cc'd a couple of people who may be more familiar with this part of the code (and hopefully able to review)

@shikharsg
Copy link
Contributor

Why isn't CI running for this PR? Can I click on "Approve and run"?

@TomAugspurger
Copy link
Contributor Author

TomAugspurger commented May 25, 2021

GitHub requires maintainer approval before running CI on PRs from first-time contributors (https://docs.github.com/en/actions/managing-workflow-runs/approving-workflow-runs-from-public-forks), to prevent abuses like mining cryptocurrencies. I thought that it was a one-time thing, and John's approval for 43cc982 would apply to this entire PR, but apparently not.

So you can click it as long as you trust I haven't inserted some crypto-mining code into the CI workflow :)

@shikharsg
Copy link
Contributor

@TomAugspurger
Copy link
Contributor Author

I merged master, in case it was just a transient issue.

@codecov
Copy link

codecov bot commented May 27, 2021

Codecov Report

Merging #759 (382d453) into master (7dc8b05) will increase coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head 382d453 differs from pull request most recent head 443968c. Consider uploading reports for the commit 443968c to get more accurate results

@@           Coverage Diff           @@
##           master     #759   +/-   ##
=======================================
  Coverage   99.94%   99.94%           
=======================================
  Files          28       28           
  Lines       10537    10562   +25     
=======================================
+ Hits        10531    10556   +25     
  Misses          6        6           
Impacted Files Coverage Δ
zarr/storage.py 100.00% <100.00%> (ø)
zarr/tests/test_core.py 100.00% <100.00%> (ø)
zarr/tests/test_hierarchy.py 100.00% <100.00%> (ø)
zarr/tests/test_storage.py 100.00% <100.00%> (ø)
zarr/tests/util.py 100.00% <100.00%> (ø)

@TomAugspurger
Copy link
Contributor Author

TomAugspurger commented May 27, 2021

Ugh, sorry, flake8 is passing now hopefully.

@TomAugspurger
Copy link
Contributor Author

The coverage issue should be fixed by 382d453 (which turned up a bug in the deprecation code)

@jakirkham
Copy link
Member

Interesting. Had to approve CI to run again. Wonder if GH forgot this was ok test

@TomAugspurger
Copy link
Contributor Author

Interesting. Had to approve CI to run again. Wonder if GH forgot this was ok test

I think someone (@shikharsg maybe) has been approving the runs on each commit. Sorry about all the failing commits, but things are green now.

@TomAugspurger
Copy link
Contributor Author

Thanks for the reviews so far. Anything else I can do to help move this along?

@joshmoore
Copy link
Member

Yeah, I've been approving the running of tests as well. I've not yet understood when it sticks and when it doesn't. (Perhaps because several of us are doing it, it's had less rhyme or reason). I'll look into a repo setting.

As for moving it along, I think it's just waiting for a review, no? Anyone in particular come to mind?

Copy link
Contributor

@shikharsg shikharsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good to go in now @TomAugspurger @joshmoore

Copy link
Member

@joshmoore joshmoore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw one minor typo in the docs, but I'll commit myself and merge. Thanks all!

@@ -11,6 +11,9 @@ Bug fixes

* FSStore: default to normalize_keys=False
By :user:`Josh Moore <joshmoore>`; :issue:`755`.
* ABSStore: compatibility with ``azure.storage.python>=12``
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for adding this!

@joshmoore joshmoore merged commit 11269a5 into zarr-developers:master Jun 1, 2021
@jakirkham
Copy link
Member

Yeah, I've been approving the running of tests as well. I've not yet understood when it sticks and when it doesn't. (Perhaps because several of us are doing it, it's had less rhyme or reason). I'll look into a repo setting.

I think it is just because Tom wasn't yet a contributor here. That should be a non-issue going forward

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ABSStore ImportError
6 participants