Skip to content

feat: Implement lock component for ProcedureManager#937

Merged
evenyag merged 5 commits intoGreptimeTeam:developfrom
evenyag:feat/procedure-manager-lock
Feb 3, 2023
Merged

feat: Implement lock component for ProcedureManager#937
evenyag merged 5 commits intoGreptimeTeam:developfrom
evenyag:feat/procedure-manager-lock

Conversation

@evenyag
Copy link
Copy Markdown
Contributor

@evenyag evenyag commented Feb 3, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

This PR introduces a lock component for ProcedureManager. To implement lock, it also needs a ProcedureMeta struct to track the metadata of a procedure in the manager.

LockMap manages all locks and it uses struct Lock to represent a lock entry.

struct Lock {
    /// Current lock owner.
    owner: ProcedureMetaRef,
    /// Waiter procedures.
    waiters: VecDeque<ProcedureMetaRef>,
}

Only the owner procedure can hold the lock. Once it releases the lock, the lock entry pops the waiter from the queue and notifies the waiter procedure.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

@evenyag evenyag marked this pull request as ready for review February 3, 2023 03:48
@codecov
Copy link
Copy Markdown

codecov bot commented Feb 3, 2023

Codecov Report

Merging #937 (2811f4c) into develop (fc9276c) will increase coverage by 0.09%.
The diff coverage is 97.31%.

@@             Coverage Diff             @@
##           develop     #937      +/-   ##
===========================================
+ Coverage    85.85%   85.95%   +0.09%     
===========================================
  Files          441      443       +2     
  Lines        59053    59202     +149     
===========================================
+ Hits         50700    50885     +185     
+ Misses        8353     8317      -36     
Flag Coverage Δ
rust 85.95% <97.31%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/common/procedure/src/lib.rs 100.00% <ø> (ø)
src/common/procedure/src/local.rs 95.00% <95.00%> (ø)
src/common/procedure/src/local/lock.rs 98.16% <98.16%> (ø)
src/object-store/src/test_util.rs 0.00% <0.00%> (-100.00%) ⬇️
src/datanode/src/instance.rs 47.11% <0.00%> (-10.67%) ⬇️
tests-integration/src/test_util.rs 80.68% <0.00%> (-8.72%) ⬇️
src/frontend/src/table/scan.rs 85.18% <0.00%> (-5.56%) ⬇️
src/servers/src/auth/user_provider.rs 81.81% <0.00%> (-2.28%) ⬇️
src/servers/src/postgres/handler.rs 82.84% <0.00%> (-0.37%) ⬇️
src/datatypes/src/data_type.rs 97.91% <0.00%> (-0.35%) ⬇️
... and 8 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Comment thread src/common/procedure/src/local.rs Outdated
Comment thread src/common/procedure/src/local.rs Outdated
Comment thread src/common/procedure/src/local/lock.rs
Comment thread src/common/procedure/src/local/lock.rs
@MichaelScofield
Copy link
Copy Markdown
Collaborator

Seems the lock implementation has some complex mechanism, specifically how the locks are propagated from parents to children. Not found in RFC. Can you give a more thorough explanation behind the design choice?

@evenyag
Copy link
Copy Markdown
Contributor Author

evenyag commented Feb 3, 2023

Not found in RFC.

See the locking part

Sub-procedures always inherit their parents' locks. The ProcedureManager only acquires locks for a procedure if its parent doesn't hold the lock.

But this PR doesn't implement this. We use this to avoid recursive locking.

@evenyag
Copy link
Copy Markdown
Contributor Author

evenyag commented Feb 3, 2023

You could refer to the PoC for the full prototype.

@MichaelScofield

Comment thread src/common/procedure/src/local/lock.rs Outdated
Comment thread src/common/procedure/src/local/lock.rs
@evenyag evenyag force-pushed the feat/procedure-manager-lock branch from 87a0809 to 2811f4c Compare February 3, 2023 09:37
@evenyag evenyag merged commit b0925d9 into GreptimeTeam:develop Feb 3, 2023
@evenyag evenyag deleted the feat/procedure-manager-lock branch February 3, 2023 10:42
paomian pushed a commit to paomian/greptimedb that referenced this pull request Oct 19, 2023
* feat: Add procedure meta

* feat: Implement lock for procedures

* chore: Allow dead code

* docs: Fix comment

* docs: Update docs of acquire_lock
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants