Cache causing deadlock

[mageddo]

What is Happening

A dead lock is occurring at the cache

thread-dump-lock.txt

Changes

• As the issue is at the cache implementation, migrate cache to ready cache impl, caffeine

Specs

• Docker Version: docker version
• DPS Version: 3.25.1

Relates to #511

[mageddo]

Issue wasn't fixed yet, the previous cache isn't the issue, the issue is the mixing of .compute and .clear methods.

All the DPS solving steps ran inside a '.compute' block which is a lock by key, eventually the block will also call '.clear' method

What is Happening

I was capable to cause deadlock using .clear() inside .compute using at least two threads, see a MRE.

Explanation

• Thread 1 acquires lock on key 1 by using computemethod
• Thread 2 acquires lock on key 2 by using computemethod
• Thread 1 call clear method to clear the cache, now waiting for thread 2 to exit compute method
• Thread 2 call clear method, now waiting on thread 1 to exit compute and clear methods
• Deadlock :/

this.store.compute(key, (k, v) -> {

  log("status=computing, key=%s", key);

  sleep(1_000);

  if (r.nextBoolean()) {
    log("status=clearing, key=%s", key);
    this.store.clear();
  }

  if (isEmpty(v)) {
    log("status=computed, key=%s", key);
    return r.nextBoolean();
  }

  return v;
});

  Number of locked synchronizers = 1
  - java.util.concurrent.ThreadPoolExecutor$Worker@378bf509

ref

Changes

• I will keep using caffeine as concurrent and memory optimized cache is a complex feature, better to use a specialized solution
• Change the caching strategy to work without or at least minimize the locks
• .clear method can be called by only one thread