-
Notifications
You must be signed in to change notification settings - Fork 4
Implement blob writing for azure storage #28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
provider_type was used twice!
I am not sure if the way the azure blob writer assigns URLs is good. But it is possible that a blob provider might store blobs under a name that si not the cid, e.g. the underlying storage does not allow assigning free names, or there are some constraints regarding directory size. So we need the ability for the azure provider to assign and return an url.
c25f3fb to
acad31b
Compare
|
Answering questions:
|
- Remove cid constraint - Add integration tests
|
Added some integration tests that just push some data to an azure bucket. They are currently ignored since you need azure credentials. |
Directory name should match crate name IMHO!
This required a signature change in the BlobWriter trait to return an Url instead of just ().
I also added a way to provide credentials via an environment variable
AZURE_STORAGE_ACCESS_KEY.Questions:
Current URL scheme:
My proposal would be to use the url field in the db only in the case where the data is not in a canonical location that the provider can figure out using the cid, and even then only add the extra info needed. So in this case url would be just the file name to allow the case where we have an azure bucket with named files that we want to content address.