S3ToSFTPOperator, SFTPToS3Operator: Support sftp_remote_host Argument

[brettpalmberg]

Description

Support new parameter sftp_remote_host in S3ToSFTPOperator and SFTPToS3Operator.

This parameter, if provided, would override remote_host stored in sftp_conn_id.

This would work the same as the remote_host parameter in SFTPOperator

Use case/motivation

This would allow storing credentials/SSH Key in a single connection. If the connection is used with multiple target hosts, remote_host could be overridden in S3ToSFTPOerator().

Example:

for h in ['host1.mydomain.com', 'host2.mydomain.com', 'host3.mydomain.com']:
    create_s3_to_sftp_job = S3ToSFTPOperator(
        task_id="create_s3_to_sftp_job",
        sftp_conn_id='sftp_conn',
        sftp_remote_host=h,
        sftp_path="/sftp_data",
        s3_bucket='my_s3_bucket',
        s3_key='/myfiles/file.txt',
    )

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[jroachgolf84]

@brettpalmberg, please feel free to open up a PR for this issue if you are interested!

[brettpalmberg]

Great, thank you. @jroachgolf84, can you assign this one to me?

[jroachgolf84]

Done!

[brettpalmberg]

Code complete. Adding a few unit tests would be good, but I do not know the testing setup well enough (yet) to make this happen.

[brettpalmberg]

@jroachgolf84, @o-nikolas, @eladkal, I believe this is code complete. Can you help move this one forward? Anything else I need to do?