Skip to content

perf(hash): use MultiGet to reduce RocksDB calls in HMSET#3327

Merged
PragmaTwice merged 6 commits into
apache:unstablefrom
sryanyuan:feat-hash-mset-multiget
Jan 7, 2026
Merged

perf(hash): use MultiGet to reduce RocksDB calls in HMSET#3327
PragmaTwice merged 6 commits into
apache:unstablefrom
sryanyuan:feat-hash-mset-multiget

Conversation

@sryanyuan
Copy link
Copy Markdown
Contributor

HMSET previously compared existing values by issuing a separate RocksDB Get
for each field. When the number of fields is large, this results in multiple
Get calls and extra overhead.

This change replaces multiple Get calls with a single MultiGet call, batching
the reads and improving performance for HMSET with many fields.

@PragmaTwice
Copy link
Copy Markdown
Member

Would you like to provide benchmark data about this change?

@sryanyuan
Copy link
Copy Markdown
Contributor Author

Would you like to provide benchmark data about this change?

In our production environment, benchmarks on real traffic demonstrate that batching reads with MultiGet in HMSET improves performance by approximately 30%.

Without MultiGet: setting 27,000 fields took ~101 ms
With MultiGet: setting 29,500 fields took ~76 ms
This optimisation reduces the overhead of multiple RocksDB Get calls and increases throughput for large hash updates.

@PragmaTwice
Copy link
Copy Markdown
Member

I should note that this structure is wrong in transaction mode:

for (...) {
  if (..) return ..;

  batch->Put(...)
}

batch->Commit()

because if early returns happen, in transaction mode we will still commit this batch in EXEC command.

But this is not introduced in this PR, so I'm fine to merge. It can be fixed later.

@PragmaTwice PragmaTwice enabled auto-merge (squash) January 7, 2026 09:51
@PragmaTwice PragmaTwice merged commit c124922 into apache:unstable Jan 7, 2026
37 checks passed
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Jan 7, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants