-
Notifications
You must be signed in to change notification settings - Fork 16.7k
Description
Apache Airflow Provider(s)
snowflake
Versions of Apache Airflow Providers
6.3.0+
Apache Airflow version
2.11
Operating System
OSX
Deployment
Astronomer
Deployment details
I'm using AwsSecretsManagerBackend
What happened
In 6.3.0, the Snowflake provider swapped to requiring base64-encoded private keys. This causes my secrets to fail as-is, since I am using plaintext encrypted private keys
My understanding is: the reason for this change was to support the GUI: multi-line password inputs are very hard to support. See #47003 for more discussion.
I use an automated system to manage secrets in a programmatic, SSoT secret store, so the GUI isn't a limiting factor for me personally. This is true of many other people. Meanwhile, b64encoding requires me to change other code that uses this as SSoT.
Update: Logic for handling private key is separate in both SnowflakeSqlApiHook and SnowflakeHook. I believe it is appropriate to consolidate the logic so that it is handled one separate place.
What you think should happen instead
I believe that it is not very difficult to support both b64encoding and plaintext. It does not need to be exclusive or.
Just check that the string starts with Update: Use try-except instead to see if it is a valid private key.-----BEGIN ENCRYPTED PRIVATE KEY-----, -----BEGIN RSA PRIVATE KEY-----, or -----BEGIN PRIVATE KEY-----. If so, then b64encoding is not being used. Else, assume b64encoding.
An alternate approach could be try: except:, but I believe it is a little more sane to just check for private key headers.
A note:
In previous discussions, I don't see anyone mentioning this as an alternative approach. I'm curious why this isn't mentioned and if I'm missing something. The previous discussions seem to focus a lot on accessing connection params via the UI, without considering other means of accessing connection params.
To me it doesn't seem like there is any reason why both ways of accessing private keys cannot be supported. But I would be interested to know if anyone else has differing opinions on this, and why base64 encoding should be the only way.
How to reproduce
Use Snowflake provider prior to 6.3, then upgrade.
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct