Skip to content

feat(mito): Init the write cache in datanode#3100

Merged
evenyag merged 17 commits into
GreptimeTeam:mainfrom
evenyag:feat/write-cache-config
Jan 9, 2024
Merged

feat(mito): Init the write cache in datanode#3100
evenyag merged 17 commits into
GreptimeTeam:mainfrom
evenyag:feat/write-cache-config

Conversation

@evenyag
Copy link
Copy Markdown
Contributor

@evenyag evenyag commented Jan 4, 2024

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

This PR initializes the WriteCache according to the mito config. By default, the write cache is under DATA_HOME/write_cache and is disabled.

It addresses an issue that the file cache doesn't use get() to access cache entries.

It also refactors some mods to make the integration easier

  • Adds a builder for the CacheManager
  • Recovers the FileCache in WriteCache::new
  • MitoEngine::new is an async function now
  • Provides a util to attach layers to ObjectStore

Now write cache is still an experimental feature so I added prefixes to its config items.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR does not require documentation updates.

Refer to a related PR or issue link (optional)

@evenyag evenyag force-pushed the feat/write-cache-config branch from 12558fc to c856ef9 Compare January 4, 2024 15:49
@evenyag evenyag changed the title feat(mito): Init the write cache feat(mito): Init the write cache in datanode Jan 5, 2024
@evenyag evenyag marked this pull request as ready for review January 5, 2024 06:17
@codecov
Copy link
Copy Markdown

codecov Bot commented Jan 5, 2024

Codecov Report

Attention: 70 lines in your changes are missing coverage. Please review.

Comparison is base (e0a43f3) 85.66% compared to head (ea008e4) 85.19%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3100      +/-   ##
==========================================
- Coverage   85.66%   85.19%   -0.48%     
==========================================
  Files         807      807              
  Lines      132167   132354     +187     
==========================================
- Hits       113227   112755     -472     
- Misses      18940    19599     +659     

Comment thread src/datanode/src/datanode.rs
@github-actions github-actions Bot added Size: L and removed Size: M labels Jan 5, 2024
@evenyag evenyag mentioned this pull request Jan 5, 2024
3 tasks
Comment thread src/mito2/src/cache.rs Outdated
Copy link
Copy Markdown
Contributor

@QuenKar QuenKar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@evenyag evenyag added this pull request to the merge queue Jan 9, 2024
Merged via the queue into GreptimeTeam:main with commit 8bd4a36 Jan 9, 2024
@evenyag evenyag deleted the feat/write-cache-config branch January 9, 2024 04:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-not-required This change does not impact docs.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants