Skip to content

feat(python): Load AWS endpoint_url using boto3 #22851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 21, 2025

Conversation

nameexhaustion
Copy link
Collaborator

@nameexhaustion nameexhaustion commented May 21, 2025

Note, this will require boto3 to be installed on the system that executes the query. It is not compatible with:

  • Passing a custom credential provider to credential_provider (or passing None)

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars labels May 21, 2025
@@ -7,4 +7,5 @@ def verbose() -> bool:
return os.getenv("POLARS_VERBOSE") == "1"


eprint = partial(print, file=sys.stderr)
def eprint(*a, **kw):
return print(*a, file=sys.stderr, **kw)
Copy link
Collaborator Author

@nameexhaustion nameexhaustion May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drive-by, ensure eprint does not capture sys.stderr upon import, this caused pytest capfd to not work as it presumably relies on patching sys.stderr (pytest-dev/pytest#5997)

Copy link

codecov bot commented May 21, 2025

Codecov Report

Attention: Patch coverage is 80.18018% with 22 lines in your changes missing coverage. Please review.

Project coverage is 81.04%. Comparing base (80e1e53) to head (1a00a38).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
crates/polars-io/src/cloud/options.rs 78.04% 9 Missing ⚠️
crates/polars-io/src/cloud/credential_provider.rs 78.78% 7 Missing ⚠️
.../polars/io/cloud/credential_provider/_providers.py 79.31% 5 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #22851      +/-   ##
==========================================
+ Coverage   80.98%   81.04%   +0.05%     
==========================================
  Files        1674     1674              
  Lines      236937   237065     +128     
  Branches     2787     2792       +5     
==========================================
+ Hits       191883   192127     +244     
+ Misses      44388    44269     -119     
- Partials      666      669       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nameexhaustion nameexhaustion marked this pull request as ready for review May 21, 2025 10:16
@ritchie46 ritchie46 merged commit 655a9c4 into pola-rs:main May 21, 2025
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature python Related to Python Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

inconsistent usage of AWS endpoint URL Scan Parquet not reading aws config to infer storage option endpoint_url
2 participants