Skip to content

Pre-buffer HTTP requests for reading parquet files.#545

Merged
delucchi-cmu merged 8 commits intomainfrom
delucchi/read_cds
Sep 25, 2025
Merged

Pre-buffer HTTP requests for reading parquet files.#545
delucchi-cmu merged 8 commits intomainfrom
delucchi/read_cds

Conversation

@delucchi-cmu
Copy link
Copy Markdown
Contributor

This enables reading non-streaming HTTP files.

It may make it slower, so we could consider adding to the "storage_options" or another UPath option.

@codecov
Copy link
Copy Markdown

codecov bot commented Sep 5, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.69%. Comparing base (a69bbd9) to head (ab9d7ec).
⚠️ Report is 66 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #545      +/-   ##
==========================================
+ Coverage   92.45%   92.69%   +0.24%     
==========================================
  Files          46       46              
  Lines        2040     2025      -15     
==========================================
- Hits         1886     1877       -9     
+ Misses        154      148       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Sep 5, 2025

Before [a69bbd9] <v0.6.5> After [7958974] Ratio Benchmark (Parameter)
36.6±0.5ms 36.8±0.7ms 1.01 benchmarks.Suite.time_pixel_tree_creation
107±0.6ms 108±0.5ms 1.01 benchmarks.time_test_alignment_even_sky
12.7±0.4ms 12.8±0.4ms 1 benchmarks.Suite.time_inner_pixel_alignment
364±2ms 363±2ms 1 benchmarks.Suite.time_outer_pixel_alignment
75.6±0.8ms 74.7±0.3ms 0.99 benchmarks.MetadataSuite.time_load_partition_info_association
19.3±0.2ms 19.2±0.1ms 0.99 benchmarks.MetadataSuite.time_load_partition_info_order6
75.3±0.3ms 74.9±1ms 0.99 benchmarks.MetadataSuite.time_load_partition_info_order7
1.07±0.03ms 1.03±0.02ms 0.96 benchmarks.time_test_cone_filter_multiple_order

Click here to view all benchmarks.

Copy link
Copy Markdown
Contributor

@gitosaurus gitosaurus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Couple of optional comments.

@delucchi-cmu delucchi-cmu requested a review from hombit September 25, 2025 14:05
Copy link
Copy Markdown
Contributor

@hombit hombit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

@delucchi-cmu delucchi-cmu merged commit 739ab85 into main Sep 25, 2025
12 of 13 checks passed
@delucchi-cmu delucchi-cmu deleted the delucchi/read_cds branch September 25, 2025 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants