Skip to content

feat: initial changes for compaction_time_window field support#1083

Merged
waynexia merged 7 commits intoGreptimeTeam:developfrom
etolbakov:feat/compaction_time_window
Apr 6, 2023
Merged

feat: initial changes for compaction_time_window field support#1083
waynexia merged 7 commits intoGreptimeTeam:developfrom
etolbakov:feat/compaction_time_window

Conversation

@etolbakov
Copy link
Copy Markdown
Collaborator

@etolbakov etolbakov commented Feb 25, 2023 •
edited
Loading

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

  • add a field compaction_time_window in TableOptions.
  • persist the inferred time window to table manifest to avoid calculating it everytime.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Related issue link

#1055

v0y4g3r
v0y4g3r reviewed Mar 9, 2023
View reviewed changes
Comment thread src/storage/src/compaction/scheduler.rs Outdated
Comment thread src/storage/src/compaction/picker.rs
etolbakov
etolbakov commented Mar 10, 2023
View reviewed changes
Comment thread src/storage/src/metadata.rs Outdated
v0y4g3r
v0y4g3r reviewed Mar 16, 2023
View reviewed changes
Comment thread src/storage/src/test_util/descriptor_util.rs Outdated
@killme2008
Copy link
Copy Markdown
Member

killme2008 commented Mar 22, 2023

hi, @etolbakov , are u still working on this PR?

@etolbakov
Copy link
Copy Markdown
Collaborator Author

etolbakov commented Mar 22, 2023

hi @killme2008
yeah, sorry it took more time than I initially though. We are in touch with @v0y4g3r on it. I think I'll be able to create a PR this weekend if that's ok?

@killme2008
Copy link
Copy Markdown
Member

killme2008 commented Mar 22, 2023

hi @killme2008 yeah, sorry it took more time than I initially though. We are in touch with @v0y4g3r on it. I think I'll be able to create a PR this weekend if that's ok?

I got it. Just a reminder. Go at your own pace. Thank you.

@etolbakov etolbakov changed the base branch from develop to admin March 27, 2023 09:37
@etolbakov etolbakov changed the base branch from admin to develop March 27, 2023 09:37
@etolbakov etolbakov marked this pull request as ready for review March 30, 2023 08:08
@etolbakov
Copy link
Copy Markdown
Collaborator Author

etolbakov commented Mar 30, 2023

Hi @v0y4g3r
Could you please take a look at the changes when you have time?

@v0y4g3r
Copy link
Copy Markdown
Contributor

v0y4g3r commented Mar 30, 2023

Hi @v0y4g3r Could you please take a look at the changes when you have time?

Sure, I will review this PR tommorow

v0y4g3r
v0y4g3r approved these changes Apr 3, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@v0y4g3r v0y4g3r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good to me. I will resolve the confilicts with upstream so that it can be merged.

Comment thread src/table/src/requests.rs
@waynexia waynexia self-requested a review April 3, 2023 06:08
@v0y4g3r
Copy link
Copy Markdown
Contributor

v0y4g3r commented Apr 3, 2023 •
edited
Loading

@etolbakov I rebased feat/compaction_time_window onto latest develop, please take another look at the changes to ensure I didn't miss something. Also, persisting inferred compaction time window is not incoperated in this PR so please add a todo in CompactionTaskImpl::write_manifest_and_apply in case we forget it.

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 3, 2023 •
edited
Loading

Codecov Report

Merging #1083 (a5b3e40) into develop (5848f27) will decrease coverage by 0.36%.
The diff coverage is 89.42%.

@@             Coverage Diff             @@
##           develop    #1083      +/-   ##
===========================================
- Coverage    85.80%   85.44%   -0.36%     
===========================================
  Files          503      506       +3     
  Lines        76227    76863     +636     
===========================================
+ Hits         65403    65675     +272     
- Misses       10824    11188     +364

waynexia
waynexia reviewed Apr 3, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@waynexia waynexia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good to me 👍

Comment thread src/table/src/requests.rs Outdated
@etolbakov
Copy link
Copy Markdown
Collaborator Author

etolbakov commented Apr 5, 2023

@v0y4g3r I've checked the changes after the rebase, LGTM.
I've added the TODO as well

@v0y4g3r v0y4g3r requested a review from waynexia April 5, 2023 10:16
waynexia
waynexia approved these changes Apr 6, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@waynexia waynexia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great, thanks @etolbakov

@waynexia waynexia merged commit 59f7630 into GreptimeTeam:develop Apr 6, 2023
@etolbakov etolbakov deleted the feat/compaction_time_window branch May 7, 2023 18:51
paomian pushed a commit to paomian/greptimedb that referenced this pull request Oct 19, 2023
@etolbakov @paomian
feat: initial changes for compaction_time_window field support (Grept…
9ae8877 
…imeTeam#1083)

* feat(compaction_time_window): initial changes for compaction_time_window field support

* feat(compaction_time_window): move PickerContext creation

* feat(compaction_time_window): update region descriptor, fix formatting

* feat(compaction_time_window): add minor enhancements

* feat(compaction_time_window): fix failing test

* feat(compaction_time_window):  return an error instead silently skip for the user provided compaction_time_window

* feat(compaction_time_window): add TODO reminder
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@v0y4g3r v0y4g3r v0y4g3r approved these changes

@waynexia waynexia waynexia approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@etolbakov @killme2008 @v0y4g3r @waynexia