Here's a blog post about global executors and adding APIs for executing futures to std:
Like the earlier blog post about async destructors, this is an area where we can make progress in improving the async/await experience the near-to-medium term without blocking on large scale language changes. But there are a lot of design questions to be sorted out as we go along here.
I'm against adding any more attributes “like the global allocator” without solving this issue at the language level. Related Portability WG issue https://github.com/rust-lang-nursery/portability-wg/issues/3 and RFC https://github.com/rust-lang/rfcs/pull/2492
I'd like to consider that claim with a bit more nuance, because I find it doesn't hold water for me.
First, let's just deal with the claim that we should block adding another #[global_*] attribute on a generalization of the problem. Here we must be careful to draw a distinction between two very different arguments:
#[global_allocator] was a fundamentally bad idea, and we should not add more APIs like that.
#[global_allocator] was a fundamentally good idea, but it should be done in a more principled, generalizable way.
These arguments are very different! The first would present a compelling case (if the underlying claim that its a bad idea has meritt) that we should not add another similar interface. But the second is not so compelling. Such a generalization already has to contend with the existence of #[global_allocator], what specific additional challenge would #[global_executor] present to generalizing this concept? That depends on the specific API, which we don't have a proposal for.
What seems much more reasonable would be to insist that #[global_executor] should not be different from #[global_allocator] in a way that makes generalizations more difficult. But assuming that holds, its just two deprecated APIs instead of one, both of which presumably are replaced with a macro expansion to the generalized language feature someday.
Second, I just don't think generalizing this concept and enabling libraries other than std to define their own of these is a very good idea. It's everything Rust's choice to use of type classes over ML modules is against.
In the course of working on Rust, we have found a few concerns which are so truly global that they justify going against the trait philosophy and adding these entry points. Memory allocation is one, I claim task execution ought to be another. But these concerns can be counted on one hand from my perspective, and I would want a lot of good practical arguments for why end users should be allowed to create more.
Is it infeasible to just explicitly pass an executor to anything spawning futures? (after adding executor traits to std or futures-rs)
It would be very inconvenient in the same way that passing an allocator to anything that allocates would be very inconvenient. Often these spawns are buried deep in routines that don't take arguments - destructors are the worst, but the point of async destructors is to keep objections from spawning tasks in their destructors.
Of course its always an option, and likely there will be use cases where that does make sense, in the same way that some in some use cases collections parameterized by an allocator make sense. This is why the futures crate provides a Spawn trait already.
Anyway, I think the big issue is that while two different subsystems/libraries using the same allocator don't interfere with each other very much, sharing an executor is much more problematic, because for instance a subsystem submitting long-running futures that doesn't care about latency is going to prevent another subsystem from getting low-latency future execution.
If we want one global executor, then it needs to be sophisticated, e.g. it must be able to detect long-running futures and futures blocked on blocking I/O and spawn additional threads when it happens.
I want to bring up the old example again: Imagine i'm writing a Bit-torrent program with GUI. So clearly there'll be something that represents a GUI event loop, and something that represents a networking event loop. Should there be one global executor for both GUI tasks and networking tasks? Or should there be two separate global executors? What should the relationships between them be?
This is interesting, since its generally been considered a problem that users might end up with two different executors running in their program (e.g. tokio and async-std). There are a few mitigations for this sort of problem where you want different tasks to be handled differently:
std::task::spawn, but instead spawn some using a different API which operates a different executor.The goal is to provide a lowest common denominator for libraries, which is "this API spawns the task onto whatever executor the end user has selected." It's then up to the end user to select good libraries and a good executor for their particular use case, as it is in all things.
I also don't think we should add a global executor. Even the post itself mentions:
Ideally, many of these library authors would not need to spawn tasks at all. Indeed, I think probably most libraries which spawn tasks should be rewritten to do something else
So motivation is already looks weak. And there is a problem of several executors used by the same app, we easily can imagine 3 executors (GUI, network, GPU), so solidifying the stance that an app should have a single blessed global executor does not look right to me.
So I think we should wait at least a year or two for Rust async ecosystem to stabilize a bit. This should allow us to crystallize pain points worth of modifying the language to solve them, otherwise this proposal looks quite premature.
A big issue I see is cancellation. If you get hold of a Future, that's easy to cancel. But if a function call spawns a task somewhere, then you lose the ability to cancel it (directly). It may keep running even after you cancel the future you've got. You may not even be able to attribute running tasks to the code they came from.
I realize that libraries can do that already by depending on an executor, but I think that is a barrier high enough that they'll do it only when it's necessary, not because it's convenient.
The question comes down to a question of structured concurrency (disclaimer: article conflict of interest in promoting author's library). I think it's generally considered bad practice to spawn a long-running thread under the caller's feet, and by analogy it should be considered bad practice to spawn a long-running task under the caller's feet. The solution is not to "add go statements" (unbounded task concurrency), but to add more structured primitives such that unbounded spawns aren't usually necessary, and for the cases where they are, passing a scoped spawn handle around makes the boundaries clear.
And I agree that any decently sized application will probably want at least two executors (even if they're powered by the same backend), one for IO-heavy tasks and one for CPU-heavy. Structured spawn handles handle this implicitly (give a handle to the correct executor), whereas a global spawn would have to know about every different entry point for differently tuned tasks.
All of that said, providing a globally available API that maintains structured scoping could be valuable! What if, for example, a GUI task wants to run some CPU-bound task then update the GUI afterwards? With a way to spawn a scoped CPU-bound task, it could just run it directly. Without, it would have to receive a bigger-scoped CPU-bound task spawn handle and make a smaller-scoped handle from it (or just spawn and await, I suppose).
The problem then again becomes enumerating the different ways one might want to tune the scoped tasks, which seems implausible.
I have few points to make here.
Allocators are a bit special, because almost anything needs them. Even panicking may allocate, if I'm not entirely mistaken. That isn't really the case for executors.
Second, it seems likely one might want to run some arbitrary code before the global executor is created ‒ argument parsing, reading configuration, configuring the executor itself ‒ number of threads and such. Having to declare the executor statically at compile time seems limiting.
Is there a reason why this shouldn't be done in a way similar to the log crate? A global variable something like EXECUTOR: OnceInit<dyn Executor> or something like that? Does it even have to live in std? If we can have random number generation and logging in a library...
If the motivation for executors to be global is that passing a context down to functions is hard, maybe Rust should find a way to pass context around easier? Such need for context passed down to functions comes up a lot in many areas, so it'd solve much more than executor instance.
Global anything comes with downsides. If allocators were passed as a context instead, we could have nice functionality for free, e.g. track how much memory a function call uses, find leaks, enforce memory limits. It'd be especially lovely for unit test keeping eye on application's memory usage.
Maybe the executor could be thread-local? At least it'd make it possible to swap "global" executor for some parts of the code.
That can be mitigated by having task::spawn return a handle which joins task in the (async) Drop. That way, tasks become more similar to allocation: everything that is leaked from function call has to be either explicitly returned to caller or explicitly mem::forgotten. But this grand plan is a pretty big chunk of under explored API area.
A minor point, but I wanted to address the comment that the GUI might require a different executor than the rest of the system (networking etc). Druid takes the position that the UI mainloop is not an async executor, but rather something that is conceptually owned by the platform, and uses synchronous callbacks to get platform events into user UI logic. While the basics of taking events and ultimately drawing onto a surface can be modeled as an async pipeline, there's a bunch of other stuff that is much cleaner if just synchronous.
I think integrating druid with the async ecosystem is interesting, but I'd be happy if spawning futures onto the UI thread is not a primitive in that integration. Nor would I want to encourage anyone to schedule, for example, a network request so that the waker is the UI thread.
Obviously this is just one of many possible choices, and probably is controversial in the Rust world. But I'm stating it as a data point of evidence that maybe we don't need multiple executors to solve this particular problem.
The possibility of multiple executors does suggest that rather than providing "spawn on the global executor" we could provide "spawn on the current executor." This is closer to how things work in existing systems (e.g. C#, Kotlin), and also closer to how the non-spawn case works in Rust.
I'm pretty concerned about there being a global executor, because I can imagine wanting multiple executors: different ones for different libraries. Note that different libraries using tokio vs async-std is different to this: I want to be able to use multiple executors, and I want to be able to control which executor each library I use uses.
To be more concrete: I don't want to use both tokio and async-std, but I might want to use tokio (default executor) + tokio-threadpool (or equiv async-std). Or I might want to use a library that had tokio or async-std in mind with GTK's event loop instead.
If some things shouldn't be using std::task:spawn, doesn't this suggest that the API isn't fit for purpose. Also, these libraries that can't use std::task:spawn are then going to be falling back to executor-specific logic. Surely the idea here is remove the need for this.
The possibility of multiple executors does suggest that rather than providing "spawn on the global executor" we could provide "spawn on the current executor."
This seems much more reasonable to me. This could exposed through something passed into the Future's poll argument right? Although, I still think we might want to distinguish between different kinds of executors. This wouldn't be a common case, but it would be nice if a sophisticated library could take multiple executors as parameters for use for different purposes.
So, I think there's a fundamental unsolvable problem with having a global executor that spawn tasks without requiring any sort of parameters, as shown by the following example:
Clearly, futures spawing by components A and B must be treated differently, but since they are both spawned by library L, it means that library L must take a parameter (explicitly or via TLS) that is then used when spawning futures so that component A and component B futures can get a different treatment.
However, if such a parameter is passed, then we might as well pass a reference to an executor along with it (or pass just a "sub-executor" instead), so there is no need for a global executor then, and thus maybe we shouldn't have one.
The other possible solution I see is to forbid component A by simply saying that it is forbidden to submit so much work to the global executor that it becomes CPU-bound, but this seems a problematic requirement, since there is no clear threshold and thus no way to ensure it is respected.
I share the concerns about global resources. There does not seem to be a good solution for dynamic libraries in place (this is a general problem for TLS/globals), and I do believe that the specific case of spawning tasks on an executor can be done much simpler by utilizing the existing task execution context. In an executor I am working on, I have an API that looks something like:
let exec = Exec::new();
exec.spawn(async {
let local_spawn = local_spawner().await;
local_spawn.spawn(async {})
});
The global function local_spawner returns a future that outputs a spawner object. It can probably be simplified to something like:
let exec = Exec::new();
exec.spawn(async {
spawn_local(async{}).await;
});
The local_spawner future uses the &mut Context parameter in the poll function to pull out a pointer to the executor from the Waker, and constructs the spawner object from that. This has the added benefit of working across dynamic library boundaries.
I think it's important that code outside of an async function is explicit about which executor it spawns on, while async functions can use its local execution context to spawn with a simpler API.
Personally I'm very excited by the prospect of having a hook to switch which global executor is used.
One of the most common questions we've received since the release of async-std is how to implement cross-runtime compatibility. And while sharing traits gets us part of the way, being able to share the executor would get us even further.
This seems like a practical solution to a practical problem. And I think it would be greatly beneficial for the async ecosystem as a whole.