Ability to have custom initialization for each worker thread

[dovahcrow]

tokio::runtime provides a builder which let users run initialization code for each worker thread. Could this be also possible in runtime, say, having an attribute proc macro runtime::initializer which can let us designate the function to be ran for worker thread initialization?

[yoshuawuyts]

@dovahcrow Interesting! -- yeah possibly!

Could you perhaps expand on what you're trying to do? I can guess to some uses for this (e.g. TLS), but having some concrete examples would help us a lot with the design. Thanks!

[dovahcrow]

@yoshuawuyts Sure. I'm currently building a latency critical system which requires me to set the CPU scheduling priority and real-time level for each worker thread. I'm doing this by calling corresponding c functions in tokio's builder:

let mut rt = Builder::new()
        .after_start(|| {
            set_self_policy(Policy::RoundRobin, 99).expect("Cannot set policy");
            set_self_affinity(CpuSet::new(1)).expect("Cannot set affinity");
        })
        .build()
        .expect("Cannot create runtime");

[yoshuawuyts]

I'm thinking there's two designs possible here:

Add fields in the entry tag:

this would likely feel the most consistent with how runtime currently works, and probably works as expected. The only thing to bikeshed would be what the name of the attribute is, and how it interacts with custom runtimes (Though I'm confident we can do it without breaking backwards compat):

#![feature(async_await, await_macro)]

fn set_affinity() {
    set_self_policy(Policy::RoundRobin, 99).expect("Cannot set policy");
    set_self_affinity(CpuSet::new(1)).expect("Cannot set affinity");
}

#[runtime::main(for_each_thread=set_affinity)]
async fn main() {
    println!("Hello world! 🤖");
}

Global

The other option is to be similar to the global allocators and introduce a new static TREADPOOL_HOOK or something that can be picked up not just by Runtime, but possibly by Rayon too.

The downside is that it can only be set once per program, but I'm thinking that might actually be a reasonable constraint for most cases.

#[global_thread_init]
static THREAD_INIT: Fn() = fn () {
    set_self_policy(Policy::RoundRobin, 99).expect("Cannot set policy");
    set_self_affinity(CpuSet::new(1)).expect("Cannot set affinity");
}

But I'm not sure how feasible the last option is.

[dovahcrow]

This first option looks really appealing to me. Actually I was thinking about having a PR using the first approach but after I took a look at how currently the runtime is implemented, I felt like there’s too much API decisions to make in order to have a PR for a contributor.

[yoshuawuyts]

@dovahcrow yeah, I feel the first option might actually be best too (:

We should probably talk about the naming too; I feel we could do better than for_each_thread. If we establish consensus on an API, would you want to make a PR?

[dovahcrow]

@yoshuawuyts Yeah absolutely! After we concluded the API design I can work on the PR, however, not the recent month. I'm afraid I don't have too much spare time recently.

[yoshuawuyts]

@dovahcrow that's all good, I really appreciate your work on this so far, and looking forward to what we can achieve here!

[sdroege]

The downside is that it can only be set once per program, but I'm thinking that might actually be a reasonable constraint for most cases.

I agree that the other API design route would seem more useful. The global allocators-like hook has exactly the problem you mention above: you might want to use different thread initialization hooks for different runtimes and also for different APIs doing things with threads. E.g. for rayon threads you might want to set priorities to optimize for throughput instead of latency, for async IO you'd favour latency, etc.

And you might even have multiple unrelated crates in your program that internally handle things via threads with one of the above and want to be able to configure things according to their specific use-case on an "internal" runtime, an internal rayon threadpool, or whatever.

PS: Slightly related, it would be nice to have APIs for explicitly instantiating and stopping runtimes (instead of having them only as a global/static thing) so they could be handled like an internal detail that does not affect other parts of the application. And also that you would be able to spawn in a dynamically loaded "plugin" that can later be unloaded again (after the runtime and everything it spawned is stopped). That's both things I'm currently doing with tokio, but I'll create a different issue for it with more details in the next days to discuss that.

[sdroege]

Related, #42