Skip to content

GH-3444: Add Custom TTL support for RedisLock, and JdbcLock #9053

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

EddieChoCho
Copy link
Contributor

@EddieChoCho EddieChoCho commented Mar 30, 2024

Fixes: #3444

  • Add DistributedLock interface which is implemented by RedisLock and JdbcLock.
  • Modify LockRegistry, ExpirableLockRegistry, RenewableLockRegistry interfaces.
  • Modify implementation of DefaultLockRepository, JdbcLockRegistry, RedisLockRegistry`
  • Modify ddl of INT_LOCK table.
  • Maintain test cases and documents.

Copy link
Member

@artembilan artembilan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this!

I still think it has to be called a DistributedLock from the beginning: we may come up eventually with other API, not only TTL.

I think the LockRegistry must be modified for a generic arg: LockRegistry<L extends Lock>.
And no need in the extra CustomTtlLockRegistry abstraction.

Can you elaborate, please, why do we need a result from delete()?

Since this is a huge braking change (as it is expected according to the discussion in the issue), we are going to look into this in details in the next 6.4 version, which we are going to start this June.

@EddieChoCho
Copy link
Contributor Author

Can you elaborate, please, why do we need a result from delete()?

If the result(row count) is 0, which means that the lock is not owned by current process.

For example:
ProcessA and processB are in the same region.
ProcessA acquired the distributed lock. //{REGION: r, LOCK_KEY: lock, CLIENT_ID: processA}
But the work takes too long and the ownership of the lock has been expired.
Before processA release the lock, processB acquired the same distributed lock. //{REGION: r, LOCK_KEY: lock, CLIENT_ID: processB}
When processA try to release the distributed lock. //DELETE FROM INT_LOCK WHERE REGION='r' AND LOCK_KEY='lock' AND CLIENT_ID='processA'
The result from delete() would be 0.

In this case, an IllegalStateException should be thrown to info the process that the integrity of data protected by this lock may have been compromised.

}
catch (TransientDataAccessException | TransactionTimedOutException | TransactionSystemException e) {
// try again
}
catch (IllegalStateException e) {
throw new IllegalStateException("Lock was released in the store due to expiration. " +
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I see why you use boolean for delete() now.
I wonder if we can use a ConcurrentModificationException instead.
Or something from the org.springframework.dao package, like ConcurrencyFailureException

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RedisLock throw IllegalStateException when it face this scenario.
Should we also modify the implementation of RedisLock?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Well, that was just a suggestion if that is makes sense to rely on some specific exception instead of this common.
Might be the fact that ConcurrentModificationException does not fit in our locks scenario.
However that ConcurrencyFailureException might be OK.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

catch (TransientDataAccessException | TransactionTimedOutException | TransactionSystemException e) {
      // try again
}

In the current implementation, TransientDataAccessException, TransactionTimedOutException, and TransactionSystemException will be caught, and then the delete() method will be invoked again.

Since ConcurrencyFailureException extends TransientDataAccessException, I think we should not throw it for this scenario.

I will change it to throw ConcurrentModificationException instead if you think ConcurrentModificationException fits better.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Thanks. Missed that.

So, yeah, this ConcurrentModificationException fits in my brain because its name means exactly what happens here for us.
But I'm not sure if this is a correct scenario where we can use it from Java perspective.
That's why I'm reaching you for other opinion, arguments and so on.
Just want to be sure that we are on the same page with a decision to move on.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see,
I agree that changing from IllegalStateException to ConcurrentModificationException can improve the readability.
But I'm also not sure if this is a correct scenario where we can use it from Java perspective.

How about we create our own exception -LockOwnershipExpiredException for Spring distributed locks?
It can be used by both RedisLock, and JdbcLock.

@artembilan
Copy link
Member

This is good stuff.
Thank you very much for looking into that!

As I expected this is really huge breaking change in the API, so I'm afraid we cannot accept it right now according to our migration policies.
We have to wait until the next major version 7.0 next year.
Of course we might be able to start it in the end of this year, when Spring Framework 6.2 release train is out this November.

Please, keep in touch meanwhile.

Thank you for understanding!

@EddieChoCho EddieChoCho marked this pull request as draft June 26, 2024 12:54
public void delete(String lock) {
this.defaultTransactionTemplate.executeWithoutResult(
transactionStatus -> this.template.update(this.deleteQuery, this.region, lock, this.id));
public boolean delete(String lock) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also presents an opportunity to check for template.update() > 1 to throw exception, this will help prevent incorrect constraint on Table.

This is possible only if table is created without constraint INT_LOCK_PK primary key (LOCK_KEY, REGION).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually outside the scope of the PR. (my bad, I should not include this change).
The method has been maintained in PR #9292.
I think business logic should not address cases of misconfiguration or incorrect settings.
If you'd like to discuss this method further, it would be great to create a new issue.

@Pavankn18
Copy link

Pavankn18 commented Oct 27, 2024

Any new on this? Spring 7 will be out soon right? Will these changes be included 7.0?

Spring framework will become GA in Nov 2024 -https://spring.io/blog/2024/10/01/from-spring-framework-6-2-to-7-0.

@artembilan
Copy link
Member

Thank you both for keeping an eye on this! So, in November we are releasing 6.4. After that we are good to start 7.0. But that is going to be released only next November 2025. So, not soon enough, but we still don’t have a choice with such a breaking change.

@artembilan
Copy link
Member

We are now at 7.0 development timeline.
I wonder if it is still OK for you to reinstate the work over here.
Thanks

@EddieChoCho EddieChoCho force-pushed the GH-3444 branch 2 times, most recently from 79a1268 to 2bab334 Compare June 9, 2025 17:20
@EddieChoCho
Copy link
Contributor Author

EddieChoCho commented Jun 9, 2025

I have rebased the branch, here is the summary of the changes:

  • RedisLock and JdbcLock now implement DistributedLock interface to support the feature of customized time-to-live for the lock status data.
  • RenewableLockRegistry now provides new renewLock(Object lockKey, long customTtl, TimeUnit customTtlTimeUnit) method, allowing user to renew the lock with a custom time-to-live value.
  • The APIs of LockRepository and DefaultLockRepository are also modified to support the feature.
  • The CREATED_DATE column in the INT_LOCK table has been replaced with EXPIRED_AFTER to support the feature.

@EddieChoCho EddieChoCho marked this pull request as ready for review June 9, 2025 17:42
@EddieChoCho EddieChoCho force-pushed the GH-3444 branch 2 times, most recently from 03b9570 to 55747ed Compare June 9, 2025 17:56
@artembilan artembilan requested a review from cppwfs June 9, 2025 19:46
…cLock

Fixes: spring-projects#3444

* Add `DistributedLock` interface which is implemented by `RedisLock` and `JdbcLock`.
* Modify `LockRegistry`, `ExpirableLockRegistry`, `RenewableLockRegistry` interfaces.
* Modify implementation of `DefaultLockRepository, `JdbcLockRegistry`, `RedisLockRegistry`
* Modify ddl of `INT_LOCK` table.
* Maintain test cases and documents.

Signed-off-by: Eddie Cho <[email protected]>
Copy link
Contributor

@cppwfs cppwfs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution.

Only had a couple of small requests.

[IMPORTANT]
====
The `CREATED_DATE` column in the `INT_LOCK` table has been replaced with `EXPIRED_AFTER` to support the custom time-to-live feature.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to have a note that states that the users will have to update their table definitions, if they are already using this feature.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if CREATED_DATE column could still be useful...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so as well. i.e. how long has this lock been around (Not just renewed).

Copy link
Contributor Author

@EddieChoCho EddieChoCho Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we keep the CREATED_DATE column, it now provides information about the time the lock was acquired, rather than the TTL.
Then we need to consider these two cases:

  1. A process acquires same lock multiple times before it releases the lock.
    In this case, once the value of CREATED_DATE column has been set during the first acquisition and should not be updated by subsequent acquisitions before the lock is released.
  2. The TTL of a lock is expires and the lock is now acquired by another process.
    In this case, sine the expired lock's data still existed in the table, we should not only update the CLIENT_ID but also the value of CREATED_DATE column.

I will modify the implementation of acquire() to cover this scenario.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one is typical for any RDBD data: when it was inserted. And we really don’t update it on subsequent acquisitions.

Copy link
Member

@artembilan artembilan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really great!
Please, take a look into distributed-locks.adoc to mention this our new DistributedLock abstraction and fix code snippet for a new style: LockRegistry<Lock> and so.

Thank you!

import java.util.concurrent.locks.Lock;

/**
* A {@link Lock} implementing for spring distributed locks
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about just to say:

A distributed {@link Lock} extension.

No need to mention Spring especially not from capital latter.

* @param customTtl the specific time-to-live for the lock status data
* @param customTtlTimeUnit the time unit of the {@code customTtl} argument
*/
void lock(long customTtl, TimeUnit customTtlTimeUnit);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use a Duration API instead?

* while acquiring the lock (and interruption of lock
* acquisition is supported)
*/
boolean tryLock(long time, TimeUnit unit, long customTtl, TimeUnit customTtlTimeUnit) throws InterruptedException;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Especially here where we have so many method arguments...
I mean Duration instead.

* @param customTtlTimeUnit the time unit of the {@code customTtl} argument
*
*/
void renewLock(Object lockKey, long customTtl, TimeUnit customTtlTimeUnit);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here about Duration.
I know tryLock(long time, TimeUnit unit) is there in Lock API, but that one has been done long time ago before java.time API.
Make sense?

* Specify the time (in milliseconds) to expire deadlocks.
* @param timeToLive the time to expire deadlocks.
*/
public void setTimeToLive(int timeToLive) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a justification of the removal of this public API?
If we are really strong enough then let's deprecate it first!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the default TTL constant from DefaultLockRepository to JdbcLockRegistry.
The TTL value can now be set through the constructor of JdbcLockRegistry, which aligns with the design of RedisLockRegistry.
As a result, I removed the setTimeToLive() method from DefaultLockRepository.

However, should I also keep a default TTL in DefaultLockRepository to support developers who use DefaultLockRepository directly without using JdbcLockRegistry?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the point.
The DefaultLockRepository really could be used (not sure why) out of JdbcLockRegistry.
This one is a public API, so as less distraction as possible.

this.ttl = DEFAULT_TTL;
}

public JdbcLockRegistry(LockRepository client, long expireAfter) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see if we can rely on Duration instead!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also update the constructor of RedisLockRegistry from
public RedisLockRegistry(RedisConnectionFactory connectionFactory, String registryKey, long expireAfter)
to
public RedisLockRegistry(RedisConnectionFactory connectionFactory, String registryKey, Duration expireAfter)
to align with the API design.

Do you agree?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure! We can do that, but as a new constructor and deprecation of existing one.
Remember, we develop here a library, therefore compatibility in API should be honored as much as possible.
Since we can still keep an old constructor here with simple adaptation, no reason to introduce another breaking change.

@@ -30,7 +30,7 @@ CREATE TABLE INT_LOCK (
LOCK_KEY CHAR(36) NOT NULL,
REGION VARCHAR(100) NOT NULL,
CLIENT_ID CHAR(36),
CREATED_DATE TIMESTAMP NOT NULL,
EXPIRED_AFTER TIMESTAMP NOT NULL,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something is off with indents.
Has to be tab.

// Make sure a transaction is active
assertThat(TransactionSynchronizationManager.isActualTransactionActive()).isTrue();

TransactionSynchronization transactionSynchronization = spy(TransactionSynchronization.class);
TransactionSynchronizationManager.registerSynchronization(transactionSynchronization);

this.client.acquire("foo"); // 1
this.client.renew("foo"); // 2
this.client.acquire("foo", Duration.ofMillis(10000)); // 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels like Duration.ofSeconds(10).
Much easier to read and less typing.
No? 😉

…cLock

Fixes: spring-projects#3444

* Modify `LockRegistry`, `DistributedLock` interfaces.
* Modify implementation of `DefaultLockRepository, `JdbcLockRegistry`, `RedisLockRegistry`
* Modify ddl of `INT_LOCK` table.
* Maintain test cases and documents.

Signed-off-by: Eddie Cho <[email protected]>
@EddieChoCho
Copy link
Contributor Author

I just pushed another commit, please review

Copy link
Member

@artembilan artembilan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see too many problem now.
Thank you for the great update!
After the next round of changes according to our review, we are going to look into merging this.

void lock(Duration ttl);

/**
* Acquires the lock with a specific time-to-live if it is free within the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method JavaDocs have to be imperative, like a command: do this, do that.
And no blank lines in method JavaDocs, please.

* while acquiring the lock (and interruption of lock
* acquisition is supported)
*/
boolean tryLock(long time, TimeUnit unit, Duration ttl) throws InterruptedException;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the first arg is not a Duration?
We always can covert it back to millis internally.
This is a new API, so let's make it consistent with end-user expectations!

* @param ttl the specific time-to-live for the lock status data
*
*/
void renewLock(Object lockKey, Duration ttl);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you have added a new API to the class, please, add your name to the @author list.

this.ttl = DEFAULT_TTL;
}

public JdbcLockRegistry(LockRepository client, Duration expireAfter) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New public API, therefore @since 7.0 and some simple JavaDoc.

@@ -85,16 +86,29 @@ protected boolean removeEldestEntry(Entry<String, JdbcLock> eldest) {

private int cacheCapacity = DEFAULT_CAPACITY;

/**
* Default value for the time-to-live property.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@since 7.0 here as well, please

* Constructs a lock registry with the supplied lock expiration.
* @param connectionFactory The connection factory.
* @param registryKey The key prefix for locks.
* @param expireAfter The expiration in {@link Duration}.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@since 7.0

}

/**
* Constructs a lock registry with the supplied lock expiration.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imperative:

Create a lock registry with the supplied lock expiration.

[source,sql]
----
ALTER TABLE INT_LOCK ADD EXPIRED_AFTER TIMESTAMP NOT NULL;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great tip!
We will add it into a Migration Guide: https://github.com/spring-projects/spring-integration/wiki

The xref:jdbc/dsl.adoc[] chapter provides more details.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, add a sentence in this New Components section about a DistributedLock API and a link to respective distributed-locks.adoc chapter.

@EddieChoCho EddieChoCho marked this pull request as draft June 12, 2025 20:52
…cLock

Fixes: spring-projects#3444

* Modify DistributedLock` interfaces.
* Modify implementation of `JdbcLockRegistry`, `RedisLockRegistry`.
* Modify JavaDoc.

Signed-off-by: Eddie Cho <[email protected]>
@EddieChoCho EddieChoCho marked this pull request as ready for review June 12, 2025 21:05
@EddieChoCho
Copy link
Contributor Author

Thanks for your feedbacks, I pushed another commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Custom TTL per LOCK in LockRegistry
4 participants