Skip to content

feat: record sst file time range in FileMeta#860

Merged
killme2008 merged 3 commits intoGreptimeTeam:developfrom
v0y4g3r:feat/sst-prune
Jan 11, 2023
Merged

feat: record sst file time range in FileMeta#860
killme2008 merged 3 commits intoGreptimeTeam:developfrom
v0y4g3r:feat/sst-prune

Conversation

@v0y4g3r
Copy link
Copy Markdown
Contributor

@v0y4g3r v0y4g3r commented Jan 10, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

First step towards support pruning SST files accroding to time range in query: extract the time ranges of SST files on flush to manifest.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

#846

@v0y4g3r v0y4g3r marked this pull request as ready for review January 11, 2023 06:57
@codecov
Copy link
Copy Markdown

codecov bot commented Jan 11, 2023

Codecov Report

Merging #860 (8e05a7a) into develop (72f05a3) will decrease coverage by 0.07%.
The diff coverage is 72.81%.

@@             Coverage Diff             @@
##           develop     #860      +/-   ##
===========================================
- Coverage    86.26%   86.19%   -0.08%     
===========================================
  Files          418      422       +4     
  Lines        54776    55793    +1017     
===========================================
+ Hits         47253    48089     +836     
- Misses        7523     7704     +181     
Flag Coverage Δ
rust 86.19% <72.81%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/storage/src/error.rs 88.88% <66.66%> (ø)
src/storage/src/sst.rs 94.62% <66.66%> (-1.13%) ⬇️
src/storage/src/sst/parquet.rs 92.99% <70.11%> (-6.11%) ⬇️
src/storage/src/flush.rs 83.57% <100.00%> (+0.60%) ⬆️
src/storage/src/manifest/test_utils.rs 100.00% <100.00%> (ø)
src/object-store/src/test_util.rs 0.00% <0.00%> (-100.00%) ⬇️
src/datanode/src/instance.rs 50.00% <0.00%> (-19.64%) ⬇️
src/meta-srv/src/handler/collect_stats_handler.rs 81.08% <0.00%> (-13.52%) ⬇️
tests-integration/src/test_util.rs 87.08% <0.00%> (-12.44%) ⬇️
src/frontend/src/table/scan.rs 85.18% <0.00%> (-5.56%) ⬇️
... and 49 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Comment thread src/storage/src/sst/parquet.rs Outdated
Copy link
Copy Markdown
Member

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@killme2008 killme2008 merged commit 4015dd8 into GreptimeTeam:develop Jan 11, 2023
paomian pushed a commit to paomian/greptimedb that referenced this pull request Oct 19, 2023
* feat: record sst file time range in FileMeta

* fix: clippy

* chore: add some log and doc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants