[Performance] Avoid lock contention for getting count of inner dictionary size

[ms-ruochenyu]

[Performance] Avoid lock contention for getting count of inner dictionary size

• You've read the Contributor Guide and Code of Conduct.
• You've included unit or integration tests for your change, where applicable.
• You've included inline docs for your change, where applicable.
• If any gains or losses in performance are possible, you've included benchmarks for your changes. More info
• There's an open issue for the PR that you are making. If you'd like to propose a new feature or change, please open an issue to discuss the change or find an existing issue.

Summary of the changes (Less than 80 chars)
Avoid lock contention for getting count of inner dictionary size in EventBasedLRUCache.

Description

We have seen bottlenecks in some of internal cloud services where EventBasedLRUCache<TKey, TValue> calls Count() for its inner _map. This property calculation brings with heavy lock contention, espeicial in SetValue() and Compact()

This change adds a new private field int this._mapSize to be updated during entries in _map being updated. _mapSize will be updated as atomic and only used to new cache size calculation so, there should not be beheavior change from observation outside.

[ms-ruochenyu]

@microsoft-github-policy-service agree company="Microsoft"

[keegan-caruso]

@ms-ruochenyu Can you please open and link an issue to this?

[brentschmaltz]

_map[key] = newCacheItem, does not imply the count has increased.

[brentschmaltz]

@ms-ruochenyu what is the performance gain?

[ms-ruochenyu]

@ms-ruochenyu what is the performance gain?

@brentschmaltz, below is a sample we collected in 1 of internal workload. The CPU cost was clustered to taking bucket write lock inside ConcurrentDictionary.Count(). So does Compact() call.

[brentschmaltz]

@ms-ruochenyu i am unable to understand your CPU diagram.

[keegan-caruso]

@ms-ruochenyu - Please open a tracking issue for this.