Skip to content

Commit 441baad

Browse files
author
Dmitry Polyakovsky
committed
valkey module rust SDK updates
Signed-off-by: dmitrypol <[email protected]> Signed-off-by: Dmitry Polyakovsky <[email protected]>
1 parent 392a849 commit 441baad

File tree

2 files changed

+273
-0
lines changed

2 files changed

+273
-0
lines changed

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,6 @@ _data/groups.json
1313
_data/resp2_replies.json
1414
_data/resp3_replies.json
1515
_data/modules.json
16+
.vscode/*
17+
.idea/*
18+
tmp/*

‎content/blog/2025-04-27-valkey-modules-rust-sdk-updates.md‎

Lines changed: 270 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,270 @@
1+
+++
2+
title= "Valkey Modules Rust SDK updates"
3+
date= 2025-05-06 01:01:01
4+
description= "Extending Valkey using Rust SDK."
5+
authors= ["dmitrypol"]
6+
categories= "modules"
7+
+++
8+
9+
In an earlier [article](/blog/modules-101/) we discussed the process of building Valkey Modules.
10+
Modules allow adding extra features (such as new commands and data types) to Valkey without making changes to the core code.
11+
We briefly touched on how to use the [Valkey Modules Rust SDK](https://github.com/valkey-io/valkeymodule-rs) to build a simple module.
12+
In this article we will expand on this topic and discuss some of the new features added to the SDK over the last year.
13+
14+
## What is the Valkey Modules Rust SDK?
15+
16+
The SDK is based on [Redis Modules Rust SDK](https://github.com/RedisLabsModules/redismodule-rs) and provides abstraction APIs on top of Valkey Modules own API.
17+
For those familiar with Rust development the SDK is a Rust crate that can be added to `Cargo.toml` file like any other Rust dependency.
18+
It requires the underlying Valkey version to have appropriate module APIs but allows writing Valkey modules in Rust, without needing to use raw pointers or unsafe code.
19+
Recently released [Bloom Filers module](/blog/introducing-bloom-filters/) is built using this crate and several of the developers who worked on the module contributed to the SDK.
20+
Let's deep dive into to new features.
21+
22+
## Client
23+
24+
`Context` struct now has several new functions to get information about the client.
25+
It provides Rust wrappers around `Module_GetClient*` functions in the underlying Valkey Module API.
26+
Most new functions return `ValkeyResult` so the module developer can handle the error appropriately.
27+
The functions can be called for the current client or by specifying `client_id`.
28+
29+
```rust
30+
fn my_client_cmd(ctx: &Context, _args: Vec<ValkeyString>) -> ValkeyResult {
31+
let client_id = ctx.get_client_id();
32+
let username = ctx.get_client_username_by_id(client_id);
33+
Ok(ValkeyValue::from(username.to_string()))
34+
}
35+
valkey_module! {
36+
...
37+
commands: [
38+
["my_client_cmd", my_client_cmd, "", 0, 0, 0],
39+
]
40+
}
41+
```
42+
43+
## Auth callback
44+
45+
Valkey 7.2 introduced support for callbacks after authentication.
46+
Now it can be leveraged via modules.
47+
48+
```rust
49+
fn auth_callback(
50+
ctx: &Context,
51+
username: ValkeyString,
52+
_password: ValkeyString,
53+
) -> Result<c_int, ValkeyError> {
54+
if ctx.authenticate_client_with_acl_user(&username) == Status::Ok {
55+
// can be combined with ctx.get_client_ip to control access by IP address
56+
let _client_ip = ctx.get_client_ip()?;
57+
...
58+
return Ok(AUTH_HANDLED);
59+
}
60+
Ok(AUTH_NOT_HANDLED)
61+
}
62+
valkey_module! {
63+
...
64+
auth: [auth_callback],
65+
}
66+
```
67+
68+
## Preload validation
69+
70+
`valkey_module!` macro already provided `init` callback that can be used to execute custom code during module load.
71+
However, `init` executes at the very end of module load after new commands and data types are created.
72+
That can be use useful but what if we wanted to stop module load before any of that happens?
73+
For example, we might need to restrict module to be loaded only on specific version of Valkey.
74+
That's where the optional `preload` can be used.
75+
76+
```rust
77+
fn preload_fn(ctx: &Context, _args: &[ValkeyString]) -> Status {
78+
let _version = ctx.get_server_version().unwrap();
79+
// respond with Status::Ok or Status::Err to prevent loading
80+
Status::Ok
81+
}
82+
valkey_module! {
83+
...
84+
preload: preload_fn,
85+
commands: [],
86+
}
87+
```
88+
89+
## Filters
90+
91+
To execute custom code before specific Valkey commands we can use command filters. This can be leveraged to replace default commands with custom comands or modify arguments.
92+
Thanks to the abstractions provided by the SDK we simply need to create a Rust function and register it in the `valkey_module!` macro.
93+
Note of caution - since filters are executed before every command this code needs to be optimized.
94+
95+
```rust
96+
fn my_filter_fn(_ctx: *mut RedisModuleCommandFilterCtx) {
97+
let cf_ctx = CommandFilterCtx::new(ctx);
98+
// check the number of arguments
99+
if cf_ctx.args_count() != 3 {
100+
return;
101+
}
102+
// get which command is being executed
103+
let _cmd = cf_ctx.cmd_get_try_as_str().unwrap();
104+
// grab various args passed to the command
105+
let _all_args = cf_ctx.get_all_args_wo_cmd();
106+
// replace command
107+
cf_ctx.arg_replace(0, "custom_cmd");
108+
}
109+
valkey_module! {
110+
...
111+
filters: [
112+
[my_filter_fn, VALKEYMODULE_CMDFILTER_NOSELF],
113+
]
114+
}
115+
```
116+
117+
## New event handlers
118+
119+
SDK now supports more server events. We can use this to execute our own code on client connect/disconnect, server shutdown or specific key events.
120+
121+
```rust
122+
#[client_changed_event_handler]
123+
fn client_changed_event_handler(_ctx: &Context, client_event: ClientChangeSubevent) {
124+
match client_event {
125+
ClientChangeSubevent::Connected => {}
126+
ClientChangeSubevent::Disconnected => {}
127+
}
128+
}
129+
#[shutdown_event_handler]
130+
fn shutdown_event_handler(_ctx: &Context, _event: u64) {
131+
...
132+
}
133+
#[key_event_handler]
134+
fn key_event_handler(ctx: &Context, key_event: KeyChangeSubevent) {
135+
match key_event {
136+
KeyChangeSubevent::Deleted => {}
137+
KeyChangeSubevent::Evicted => {}
138+
KeyChangeSubevent::Overwritten => {}
139+
KeyChangeSubevent::Expired => {}
140+
}
141+
}
142+
```
143+
144+
## Custom ACL categories support
145+
146+
Valkey 8 introduced support for custom ACL categories. To implement that we need to enable `required-features = ["min-valkey-compatibility-version-8-0"]` in `Cargo.toml` and register new categories in `valkey_module!` macro. Then we can restrict our custom commands to these ACL categories.
147+
148+
```rust
149+
valkey_module! {
150+
...
151+
acl_categories: [
152+
"acl_one",
153+
"acl_two"
154+
]
155+
commands: [
156+
["cmd1", cmd1, "write", 0, 0, 0, "acl_one"],
157+
["cmd2", cmd2, "", 0, 0, 0, "acl_one acl_two"],
158+
]
159+
```
160+
161+
## Validating / Rejecting CONFIG SET
162+
163+
SDK now supports specifying optional callback functions to validate or reject custom configuations. This is available for String, i64, bool and enum configs. Here is an example of validation for i64 custom config.
164+
165+
```rust
166+
lazy_static! {
167+
static ref CONFIG_I64: ValkeyGILGuard<i64> = ValkeyGILGuard::default();
168+
}
169+
fn on_i64_config_set<G, T: ConfigurationValue<i64>>(
170+
config_ctx: &ConfigurationContext,
171+
_name: &str,
172+
val: &'static T,
173+
) -> Result<(), ValkeyError> {
174+
if val.get(config_ctx) == custom_logic_here {
175+
log_notice("log message here");
176+
Err(ValkeyError::Str("error message here "))
177+
} else {
178+
Ok(())
179+
}
180+
}
181+
valkey_module! {
182+
configurations: [
183+
i64: [
184+
["my_i64", &*CONFIG_I64, 10, 0, 1000, ConfigurationFlags::DEFAULT, Some(Box::new(on_configuration_changed)), Some(Box::new(on_i64_config_set::<ValkeyString, ValkeyGILGuard<i64>>))],
185+
],
186+
...
187+
]
188+
}
189+
```
190+
191+
## Defrag
192+
193+
There is a new `Defrag` struct abstracting away the raw C FFI calls to implement defrag for custom data types.
194+
195+
```rust
196+
static MY_VALKEY_TYPE: ValkeyType = ValkeyType::new(
197+
"mytype123",
198+
0,
199+
raw::RedisModuleTypeMethods {
200+
...
201+
defrag: Some(defrag),
202+
},
203+
);
204+
unsafe extern "C" fn defrag(
205+
defrag_ctx: *mut raw::RedisModuleDefragCtx,
206+
_from_key: *mut RedisModuleString,
207+
value: *mut *mut c_void,
208+
) -> i32 {
209+
let defrag = Defrag::new(defrag_ctx);
210+
...
211+
0
212+
}
213+
valkey_module! {
214+
...
215+
data_types: [
216+
MY_VALKEY_TYPE,
217+
],
218+
...
219+
}
220+
```
221+
222+
## Redis support
223+
224+
There is a new feature flag if your module needs to run on both Valkey and Redis.
225+
Specify `use-redismodule-api` so that module used RedisModule API Initialization for backwards compatibility.
226+
227+
```rust
228+
[features]
229+
use-redismodule-api = ["valkey-module/use-redismodule-api"]
230+
default = []
231+
232+
cargo build --release --features use-redismodule-api
233+
```
234+
235+
## Unit tests and memory allocation
236+
237+
This feature flag supports running unit tests outside of Valkey or Redis. As the result it cannot use Vakey Allocator and instead relies on the System Allocator instead. The core logis is present in `alloc.rs` but developers only need to specify this in the module `Cargo.toml`.
238+
239+
```rust
240+
[features]
241+
enable-system-alloc = ["valkey-module/system-alloc"]
242+
243+
cargo test --features enable-system-alloc
244+
```
245+
246+
## Ideas for future development
247+
248+
* Mock Context for unit testing - enable unit testing functions that require `&ctx`
249+
* Context in Filters - enable more operations to be done in filters.
250+
* Environment specific config support - during development or unit testing it is useful to have different config settings.
251+
* Crontab - `cron_event_handler` uses `serverCron` (10 times per second by default) but it can be extended to run custom code on pre-determined schedule and configured via custom commands.
252+
* And more. Please stay tuned.
253+
254+
## List of contributors
255+
256+
Below are the Github handles of the developers who contibuted to the SDK in the past year
257+
258+
* [KarthikSubbarao](https://github.com/KarthikSubbarao)
259+
* [dmitrypol](https://github.com/dmitrypol)
260+
* [sachinvmurthy](https://github.com/sachinvmurthy)
261+
* [zackcam](https://github.com/zackcam)
262+
* [YueTang-Vanessa](https://github.com/YueTang-Vanessa)
263+
* [hahnandrew](https://github.com/hahnandrew)
264+
* [Mkmkme](https://github.com/Mkmkme)
265+
266+
## Usefull links
267+
268+
* [Valkey repo](https://github.com/valkey-io/valkey)
269+
* [Valkey Modules Rust SDK](https://github.com/valkey-io/valkeymodule-rs)
270+
* [Rust in VS Code](https://code.visualstudio.com/docs/languages/rust)

0 commit comments

Comments
 (0)