Skip to content

Commit 68ce25c

Browse files
dmitrypolmadolsonphiro56Dmitry Polyakovsky
authored
Blog post describing updates to Valkey Modules Rust SDK (#250)
### Description Blog post describing updates to Valkey Modules Rust SDK ### Issues Resolved <!-- List any issues this PR will resolve. --> <!-- Example: closes #1234 --> ### Check List - [ ] Commits are signed per the DCO using `--signoff` By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License. --------- Signed-off-by: Dmitry Polyakovsky <[email protected]> Signed-off-by: Daniel Phillips <[email protected]> Signed-off-by: dmitrypol <[email protected]> Co-authored-by: Madelyn Olson <[email protected]> Co-authored-by: Daniel Phillips <[email protected]> Co-authored-by: Dmitry Polyakovsky <[email protected]>
1 parent 0a0e48a commit 68ce25c

File tree

2 files changed

+279
-0
lines changed

2 files changed

+279
-0
lines changed

‎.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,6 @@ _data/groups.json
1313
_data/resp2_replies.json
1414
_data/resp3_replies.json
1515
_data/modules.json
16+
.vscode/*
17+
.idea/*
18+
tmp/*

‎content/blog/2025-04-27-valkey-modules-rust-sdk-updates.md

Lines changed: 276 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,276 @@
1+
+++
2+
title= "Valkey Modules Rust SDK updates"
3+
date= 2025-05-20 01:01:01
4+
description= "Extending Valkey using Rust SDK."
5+
authors= ["dmitrypol"]
6+
categories= "modules"
7+
+++
8+
9+
In an earlier [article](/blog/modules-101/) we explored the process of building Valkey Modules to enable developers to add features such as new commands and data types to Valkey without modifying its core.
10+
We also introduced the [Valkey Modules Rust SDK](https://github.com/valkey-io/valkeymodule-rs) demonstrating how to use it to create a basic module.
11+
In this follow-up article, we’ll dive deeper into the SDK and highlight several new features and improvements introduced over the past year.
12+
This article assumes that the reader is well familiar with Rust and Valkey modules.
13+
14+
## What is the Valkey Modules Rust SDK?
15+
16+
The SDK is based on [Redis Modules Rust SDK](https://github.com/RedisLabsModules/redismodule-rs) and provides abstraction APIs on top of Valkey's own modules API.
17+
For those familiar with Rust development the SDK is a Rust crate that can be added to `Cargo.toml` file like any other Rust dependency.
18+
It requires the underlying Valkey server to have appropriate module APIs but allows writing Valkey modules in Rust, without needing to use raw pointers or unsafe code.
19+
The recently released [Bloom Filters module](/blog/introducing-bloom-filters/) is built using this SDK and several of the developers who worked on the module contributed to the SDK.
20+
Let's deep dive into the new features.
21+
22+
## Client
23+
24+
We begin with enhancements that give developers deeper insight into the clients connected to Valkey.
25+
The `Context` struct has been extended with several new functions that allow retrieving client specific data, such as client name, username or IP address.
26+
It provides Rust wrappers around `Module_GetClient*` functions in the underlying Valkey Module API.
27+
Most of these new functions return `ValkeyResult` so that the module developer can handle the error appropriately.
28+
The functions can be called for the current client or by specifying `client_id`.
29+
30+
```rust
31+
fn my_client_cmd(ctx: &Context, _args: Vec<ValkeyString>) -> ValkeyResult {
32+
let client_id = ctx.get_client_id();
33+
let username = ctx.get_client_username_by_id(client_id);
34+
Ok(ValkeyValue::from(username.to_string()))
35+
}
36+
valkey_module! {
37+
...
38+
commands: [
39+
["my_client_cmd", my_client_cmd, "", 0, 0, 0],
40+
]
41+
}
42+
```
43+
44+
## Auth callback
45+
46+
Another key improvement is support for authentication callbacks introduced in Valkey 7.2.
47+
Thanks to the new features in the SDK Rust modules can now integrate directly with Valkey's auth flow, making it possible to implement custom authentication logic or enhance security policies on a per-client basis.
48+
One potential use case is to combine it with `ctx.get_client_ip` function described above to allow some users access only from specific IP addresses.
49+
50+
```rust
51+
fn auth_callback(
52+
ctx: &Context,
53+
username: ValkeyString,
54+
_password: ValkeyString,
55+
) -> Result<c_int, ValkeyError> {
56+
if ctx.authenticate_client_with_acl_user(&username) == Status::Ok {
57+
let _client_ip = ctx.get_client_ip()?;
58+
...
59+
return Ok(AUTH_HANDLED);
60+
}
61+
Ok(AUTH_NOT_HANDLED)
62+
}
63+
valkey_module! {
64+
...
65+
auth: [auth_callback],
66+
}
67+
```
68+
69+
## Preload validation
70+
71+
While the `valkey_module!` macro already provided an `init` callback to execute custom code during module load, it executed at the very end of the module load after new commands and data types were created.
72+
That can be useful but what if we wanted to stop module load before any of that happened?
73+
For example, we might need to restrict a module to be loaded only on specific version of Valkey.
74+
That's where the optional `preload` comes in.
75+
76+
```rust
77+
fn preload_fn(ctx: &Context, _args: &[ValkeyString]) -> Status {
78+
let _version = ctx.get_server_version().unwrap();
79+
// respond with Status::Ok or Status::Err to prevent loading
80+
Status::Ok
81+
}
82+
valkey_module! {
83+
...
84+
preload: preload_fn,
85+
commands: [],
86+
}
87+
```
88+
89+
## Filters
90+
91+
To execute custom code before specific Valkey commands we can use command filters which is now supported in the SDK.
92+
Filters can be leveraged to replace default commands with custom commands or modify arguments.
93+
Thanks to the abstractions provided by the SDK we simply need to create a Rust function and register it in the `valkey_module!` macro.
94+
Note of caution - since filters are executed before every command this code needs to be optimized for performance.
95+
96+
```rust
97+
fn my_filter_fn(_ctx: *mut RedisModuleCommandFilterCtx) {
98+
let cf_ctx = CommandFilterCtx::new(ctx);
99+
// check the number of arguments
100+
if cf_ctx.args_count() != 3 {
101+
return;
102+
}
103+
// get which command is being executed
104+
let _cmd = cf_ctx.cmd_get_try_as_str().unwrap();
105+
// grab various args passed to the command
106+
let _all_args = cf_ctx.get_all_args_wo_cmd();
107+
// replace command
108+
cf_ctx.arg_replace(0, "custom_cmd");
109+
}
110+
valkey_module! {
111+
...
112+
filters: [
113+
[my_filter_fn, VALKEYMODULE_CMDFILTER_NOSELF],
114+
]
115+
}
116+
```
117+
118+
## New event handlers
119+
120+
Reacting to server events can be very important to the module behavior.
121+
The SDK has expanded its support for registering event handlers, allowing developers to hook into more server-side events.
122+
We can use this to execute our own code on client connect/disconnect, server shutdown or specific key events.
123+
124+
```rust
125+
#[client_changed_event_handler]
126+
fn client_changed_event_handler(_ctx: &Context, client_event: ClientChangeSubevent) {
127+
match client_event {
128+
ClientChangeSubevent::Connected => {}
129+
ClientChangeSubevent::Disconnected => {}
130+
}
131+
}
132+
#[shutdown_event_handler]
133+
fn shutdown_event_handler(_ctx: &Context, _event: u64) {
134+
...
135+
}
136+
#[key_event_handler]
137+
fn key_event_handler(ctx: &Context, key_event: KeyChangeSubevent) {
138+
match key_event {
139+
KeyChangeSubevent::Deleted => {}
140+
KeyChangeSubevent::Evicted => {}
141+
KeyChangeSubevent::Overwritten => {}
142+
KeyChangeSubevent::Expired => {}
143+
}
144+
}
145+
```
146+
147+
## Custom ACL categories support
148+
149+
Valkey 8 introduced support for custom ACL categories which simplifies access control for custom commands introduced in a module.
150+
To implement that we need to enable `required-features = ["min-valkey-compatibility-version-8-0"]` in `Cargo.toml` and register new categories in `valkey_module!` macro.
151+
Then we can restrict our custom commands to these custom ACL categories.
152+
153+
```rust
154+
valkey_module! {
155+
...
156+
acl_categories: [
157+
"acl_one",
158+
"acl_two"
159+
]
160+
commands: [
161+
["cmd1", cmd1, "write", 0, 0, 0, "acl_one"],
162+
["cmd2", cmd2, "", 0, 0, 0, "acl_one acl_two"],
163+
]
164+
```
165+
166+
## Validating / Rejecting CONFIG SET
167+
168+
Configuration flexibility is important but so is validation.
169+
The SDK now supports specifying optional callback functions to validate or reject custom configurations.
170+
This is available for `String`, `i64`, `bool` and `enum` configs.
171+
Here is an example of such validation for `i64` custom config.
172+
173+
```rust
174+
lazy_static! {
175+
static ref CONFIG_I64: ValkeyGILGuard<i64> = ValkeyGILGuard::default();
176+
}
177+
fn on_i64_config_set<G, T: ConfigurationValue<i64>>(
178+
config_ctx: &ConfigurationContext,
179+
_name: &str,
180+
val: &'static T,
181+
) -> Result<(), ValkeyError> {
182+
if val.get(config_ctx) == custom_logic_here {
183+
log_notice("log message here");
184+
Err(ValkeyError::Str("error message here "))
185+
} else {
186+
Ok(())
187+
}
188+
}
189+
valkey_module! {
190+
configurations: [
191+
i64: [
192+
["my_i64", &*CONFIG_I64, 10, 0, 1000, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed)), Some(Box::new(on_i64_config_set::<ValkeyString, ValkeyGILGuard<i64>>))],
193+
],
194+
...
195+
]
196+
}
197+
```
198+
199+
## Defrag
200+
201+
For memory-sensitive applications, defragmentation is essential.
202+
The SDK now offers a safe and idiomatic Rust abstraction over the defrag API for custom data types.
203+
The new `Defrag` struct abstracts away the raw C FFI calls.
204+
205+
```rust
206+
static MY_VALKEY_TYPE: ValkeyType = ValkeyType::new(
207+
"mytype123",
208+
0,
209+
raw::RedisModuleTypeMethods {
210+
...
211+
defrag: Some(defrag),
212+
},
213+
);
214+
unsafe extern "C" fn defrag(
215+
defrag_ctx: *mut raw::RedisModuleDefragCtx,
216+
_from_key: *mut RedisModuleString,
217+
value: *mut *mut c_void,
218+
) -> i32 {
219+
let defrag = Defrag::new(defrag_ctx);
220+
...
221+
0
222+
}
223+
valkey_module! {
224+
...
225+
data_types: [
226+
MY_VALKEY_TYPE,
227+
],
228+
...
229+
}
230+
```
231+
232+
## Redis support
233+
234+
Need your module to work with both Valkey and recent versions of Redis?
235+
The SDK includes a compatibility feature flag to ensure your module runs on both Valkey and Redis.
236+
Specify `use-redismodule-api` so that module used RedisModule API Initialization for backwards compatibility.
237+
238+
```rust
239+
[features]
240+
use-redismodule-api = ["valkey-module/use-redismodule-api"]
241+
default = []
242+
243+
cargo build --release --features use-redismodule-api
244+
```
245+
246+
## Unit tests and memory allocation
247+
248+
This feature enables writing unit tests to run outside of Valkey or Redis.
249+
Instead of using Valkey Allocator it relies on the System Allocator.
250+
Unit tests allow us to perform much more granular testing and execute much faster.
251+
The core logic lives in `alloc.rs` but developers only need to specify this feature in the module `Cargo.toml`.
252+
253+
```rust
254+
[features]
255+
enable-system-alloc = ["valkey-module/system-alloc"]
256+
257+
cargo test --features enable-system-alloc
258+
```
259+
260+
## Conclusion
261+
262+
The Valkey Modules Rust SDK has seen exciting improvements over the past year, making it easier and more powerful to extend Valkey.
263+
But we are not stopping.
264+
Some of the ideas for future development include [mock context support for unit testing](https://github.com/valkey-io/valkeymodule-rs/issues/202), [enhanced context access within filters](https://github.com/valkey-io/valkeymodule-rs/issues/203), and [environment specific configs](https://github.com/valkey-io/valkeymodule-rs/issues/204) to streamline development and testing.
265+
Additionally, the introduction of [crontab scheduling](https://github.com/valkey-io/valkeymodule-rs/issues/205) will allow executing custom logic on a defined schedules using `cron_event_handler`.
266+
267+
We hope this overview helped you understand how to leverage the SDK and inspired you to explore what's possible with Valkey modules.
268+
Stay tuned for future updates.
269+
270+
We also want to express appreciation to the engineers who contributed to the SDK in the past year - [KarthikSubbarao](https://github.com/KarthikSubbarao), [dmitrypol](https://github.com/dmitrypol), [sachinvmurthy](https://github.com/sachinvmurthy), [zackcam](https://github.com/zackcam), [YueTang-Vanessa](https://github.com/YueTang-Vanessa), [hahnandrew](https://github.com/hahnandrew), [Mkmkme](https://github.com/Mkmkme).
271+
272+
## Useful links
273+
274+
* [Valkey repo](https://github.com/valkey-io/valkey)
275+
* [Valkey Modules Rust SDK](https://github.com/valkey-io/valkeymodule-rs)
276+
* [Rust in VS Code](https://code.visualstudio.com/docs/languages/rust)

0 commit comments

Comments
 (0)