-
Notifications
You must be signed in to change notification settings - Fork 356
Fix prefix match for ADLS_SAS_TOKEN #2461
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix prefix match for ADLS_SAS_TOKEN #2461
Conversation
@@ -207,7 +207,7 @@ def _adls(properties: Properties) -> AbstractFileSystem: | |||
from azure.core.credentials_async import AsyncTokenCredential | |||
|
|||
for key, sas_token in { | |||
key.replace(f"{ADLS_SAS_TOKEN}.", ""): value for key, value in properties.items() if key.startswith(ADLS_SAS_TOKEN) | |||
key.replace(f"{ADLS_SAS_TOKEN}.", ""): value for key, value in properties.items() if key.startswith(f"{ADLS_SAS_TOKEN}.") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like the fix for the bug 👍, the Java side gets the props like this with the ADLS_SAS_TOKEN_PREFIX
including the .
.
@kevinjqliu @Fokko, please may you take a look, so the client still works if Catalogs set the refresh property? (I see that the 0.10 vote just passed so suppose this'd have to be for the next release 😄) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oof, this is a good catch, @tom-s-powell. Would it be wise to throw in a test to prevent us from breaking this in the future?
Added a test @Fokko if this gets at what you were thinking? |
@tom-s-powell Yes, that looks great. Thanks! |
Thanks for the PR @tom-s-powell and @smaheshwar-pltr @Fokko for the review :) |
The current logic when constructing an ADLS file system incorrectly matches
adls.sas-token-expires-at-ms.
prefixed keys. If this is provided and encountered prior toadls.sas-token.
prefixed keys, it will incorrectly determine theADLS_ACCOUNT_NAME
andADLS_SAS_TOKEN
.For example with the following properties:
Before
After