`Executor::spin()` spawns a new OS thread per call, causing severe performance overhead

[azerupi]

Summary

Every call to Executor::spin() spawns a new OS thread for the WaitSetRunner, even for short-lived spins like spin(timeout=0). This makes the common pattern of calling spin() in a tight loop (equivalent to rclcpp's spin_some) extremely expensive, roughly ~9x slower than running the wait set on the calling thread.

Minimal reproduction

use rclrs::{Context, CreateBasicExecutor, IntoPrimitiveOptions, QoSProfile, SpinOptions};
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};

fn main() {
    let context = Context::default();
    let mut executor = context.create_basic_executor();
    let node = executor.create_node("bench").unwrap();

    let qos = QoSProfile::default().reliable().keep_last(1000);
    let recv_count = Arc::new(AtomicU64::new(0));

    let pub_handle = node
        .create_publisher::<std_msgs::msg::UInt8MultiArray>("bench_topic".qos(qos.clone()))
        .unwrap();

    let count = Arc::clone(&recv_count);
    let _sub = node
        .create_subscription::<std_msgs::msg::UInt8MultiArray, _>(
            "bench_topic".qos(qos.clone()),
            move |_msg: std_msgs::msg::UInt8MultiArray| {
                count.fetch_add(1, Ordering::Relaxed);
            },
        )
        .unwrap();

    // Publish-then-spin loop (same pattern as rclcpp benchmarks)
    let mut sent: u64 = 0;
    let start = Instant::now();
    let duration = Duration::from_secs(5);

    while start.elapsed() < duration {
        let msg = std_msgs::msg::UInt8MultiArray::default();
        pub_handle.publish(msg).ok();
        sent += 1;

        // Process available callbacks (this is the bottleneck)
        executor.spin(SpinOptions::new().timeout(Duration::ZERO));
    }

    let received = recv_count.load(Ordering::Relaxed);
    let elapsed = start.elapsed().as_secs_f64();
    eprintln!("Sent:       {sent}");
    eprintln!("Received:   {received}");
    eprintln!("Throughput: {:.0} msg/s", received as f64 / elapsed);
}

Equivalent rclcpp benchmark

#include <rclcpp/rclcpp.hpp>
#include <std_msgs/msg/u_int8_multi_array.hpp>

#include <atomic>
#include <chrono>
#include <cstdio>

int main(int argc, char** argv) {
    rclcpp::init(argc, argv);

    auto executor = std::make_shared<rclcpp::executors::SingleThreadedExecutor>();
    auto node = std::make_shared<rclcpp::Node>("spin_bench");
    executor->add_node(node);

    auto qos = rclcpp::QoS(1000).reliable();
    std::atomic<uint64_t> recv_count{0};

    auto pub_handle = node->create_publisher<std_msgs::msg::UInt8MultiArray>("bench_topic", qos);

    auto sub = node->create_subscription<std_msgs::msg::UInt8MultiArray>(
        "bench_topic", qos,
        [&recv_count](const std_msgs::msg::UInt8MultiArray::SharedPtr) {
            recv_count.fetch_add(1, std::memory_order_relaxed);
        });

    uint64_t sent = 0;
    auto start = std::chrono::steady_clock::now();
    auto duration = std::chrono::seconds(5);

    while (std::chrono::steady_clock::now() - start < duration) {
        std_msgs::msg::UInt8MultiArray msg;
        pub_handle->publish(msg);
        sent++;

        executor->spin_some(std::chrono::milliseconds(0));
    }

    uint64_t received = recv_count.load();
    double elapsed = std::chrono::duration<double>(
        std::chrono::steady_clock::now() - start).count();

    fprintf(stderr, "Sent:       %lu\n", sent);
    fprintf(stderr, "Received:   %lu\n", received);
    fprintf(stderr, "Throughput: %.0f msg/s\n", received / elapsed);

    rclcpp::shutdown();
    return 0;
}

Results (release build, 3 runs averaged, same machine)

Implementation	Throughput
rclcpp spin_some	~445k msg/s
rclrs spin() (current)	~42k msg/s
rclrs spin_some() (see below)	~374k msg/s

The current spin() is 10.6x slower than rclcpp.

Root cause

BasicExecutorRuntime::spin() does the following on every call:

1. Drains all runners and sends them through a futures::channel::mpsc channel
2. Creates a manage_workers async task
3. Spawns a new OS thread via std::thread::spawn() inside WaitSetRunner::run()
4. The main thread polls async tasks until the runner thread finishes
5. The runner thread sends results back through a blocking channel
6. The main thread joins the results and stores runners back

For a spin(timeout=0) call, the runner thread simply calls rcl_wait(0) (which returns immediately), processes any ready entities, and exits. The actual work takes microseconds, but the thread lifecycle overhead dominates:

• std::thread::spawn(): ~10-50 us on Linux
• Channel send/receive: additional synchronization overhead
• manage_workers async task creation and polling: additional overhead

In contrast, rclcpp's spin_some() runs everything on the calling thread: direct function calls to rcl_wait, readiness checks, and callback execution. No threads, no channels, no async machinery.

Validation

To confirm that thread-per-spin is the bottleneck, I implemented an experimental spin_some() that calls WaitSetRunner::run_blocking() directly on the calling thread, bypassing thread creation entirely. With that change, throughput went from ~42k to ~374k msg/s, closing the gap to only 1.2x slower than rclcpp (see table above).

Impact

This affects any application that calls spin() repeatedly with short timeouts, which is the standard ROS 2 pattern for non-blocking spinning.

[mxgrey]

Thanks for creating this benchmark and raising this concern.

It's true that spawning and destroying a thread for the WaitSetRunner is going to create unreasonable overhead for the particular usage pattern that you're describing here. This is definitely worth discussing so we can refine how the basic executor (and other executors in the future) are implemented, to balance the "principle of least surprise" against the complexity of the implementation and any other concerns around performance and resource utilization.

This affects any application that calls spin() repeatedly with short timeouts, which is the standard ROS 2 pattern for non-blocking spinning.

I think the first thing we should try to clarify is whether it makes sense for users to transplant how "non-blocking spinning" is done from rclcpp to rclrs. There are many practices that are common in C++ which would be considered bad practices in Rust. In this case, Rust offers certain patterns around concurrency which simply don't exist in C++. I think it's worth scrutinizing whether this "non-blocking spinning" pattern is something we should account for or something that should be discouraged for rclrs.

In my mind, an rclrs application should block on spinning the executor. That spinning should be seen as the main event loop, and any other activity that the user wants to interlace with execution should be spawned as tasks using ExecutorCommands::run or by creating a timer.

I can't think of a good reason to interlace calls of Executor::spin with other code as part of a higher-level main event loop. If the user wants to halt or continue the spinning based on some condition, they can use ExecutorCommands::halt_spinning to stop the spinning at any time.

The process of starting and then stopping the spinning has more sources of overhead than just spawning the thread. Before the executor returns from the spin function it needs to synchronize all the worker threads. An important guarantee that the Executor makes is that there are no ROS callbacks being executed by the time spin returns.

But suppose we want to support this usage pattern anyway

Whether or not I personally think it's the "right" pattern for rclrs users, it's worth thinking through what it would take for the pattern to be well supported.

The most straightforward approach I can think of would be for the executor to maintain a thread pool of every thread that it has needed to spawn, and then recycle those threads in between calls to Executor::spin. This would raise the following questions in my mind:

1. Would it be detrimental to the overall OS thread scheduling to have these idle threads? Presumably in between calls to spin, the threads would be waiting on a condition variable to wake them up. Would this arrangement waste CPU or OS resources if there is a significant gap between calls to spin?
2. Would waiting on a condition variable between spins be substantially better performance than spawning and despawning threads? I suspect the answer to this is yes since I believe this is what most thread pool implementations do.
3. What does the multi-threaded executor in rclcpp do about this problem? I would guess it also uses a thread pool.

@azerupi, do you have any alternative suggestions for how to smooth out the performance when transitioning in and out of spins?

[azerupi]

Thank you for your thoughtful reply!
To be honest, I'm not sure what the best path forward is, which is why I tried to not make any recommendations and just start a discussion around this. I implemented a hack just to prove out the issue had been identified correctly, but it is not the right solution. Given we have the SpinOptions that take on this role adding a spin_some doesn't fit well.

I think the first thing we should try to clarify is whether it makes sense for users to transplant how "non-blocking spinning" is done from rclcpp to rclrs. There are many practices that are common in C++ which would be considered bad practices in Rust. In this case, Rust offers certain patterns around concurrency which simply don't exist in C++. I think it's worth scrutinizing whether this "non-blocking spinning" pattern is something we should account for or something that should be discouraged for rclrs.

I don't necessarily disagree with you on this. But there are a lot of patterns that people will try to carry over from C++, like in this instance. And I feel like we either need to have a really good story around recommending and guiding user to alternative patterns or we need to support the same patterns to the extent possible. Otherwise we could end up with a lot of disappointment and confusion when people try out rclrs.

I can't think of a good reason to interlace calls of Executor::spin with other code as part of a higher-level main event loop. If the user wants to halt or continue the spinning based on some condition, they can use ExecutorCommands::halt_spinning to stop the spinning at any time.

I think a major difficulty we are going to hit here is the visibility. How do we guide new users to those patterns before they hit the issues that are caused by transposing the rclcpp patterns to rclrs.

[mxgrey]

I feel like we either need to have a really good story around recommending and guiding user to alternative patterns or we need to support the same patterns to the extent possible. Otherwise we could end up with a lot of disappointment and confusion when people try out rclrs.

I 💯% agree with you on this. As long as there isn't any major downside to improving the performance of this usage pattern, we should endeavor to make it work as effectively as users would expect it to, otherwise we would be violating the principle of least surprise.

If no one can think of any particular problem with the thread pool approach that I proposed, then I can try to find time to experiment with that.