How often do you want non-send futures?

[nikomatsakis]

So I recently became aware of something that I had not considered regarding async fn – in particular, the current default where we don’t require the resulting future to be Send doesn’t work out so well once we scale up to async fn in traits. (I take it many people are aware of this for some time, but I at least hadn’t thought about it before and I’d like to talk it out.)

The problem

The current async fn definition returns an impl Future – it does not promise that the resulting future is send. In practice, this is fine, because auto traits will automatically figure out whether it should be send or not. Very ergonomic, very nice:

async fn process() { }

fn foo() {
  spawn(process()); // requires a sendable future; works b/c auto-traits
}

However, there is a wrinkle. When we support async fn in traits, things won’t work out as well:

trait Process {
  async fn process(&self);
}

fn foo<P: Process>(p: P) {
    let x = spawn(p.process()); // ERROR
}

The problem here is that the definition of foo must be valid for all processes Process types, including those whose resulting future is not send.

Options

If we stick with the current design, we can resolve this in three ways (detailed below):

• Some form of bound that foo can use to specify that P::process returns a Send future
• Some way to opt-in to sendability async(Send) fn process()
• A different default in traits (e.g., async fns in traits are send by default)

Alternatively, if we change the design we could:

• default to sendable futures, but offer an opt-out like async(?Send).

None of these options look that appealing to me right now. They all seem to impose a cost somewhere.

Questions

I have a few questions I would like feedback on:

• How often will people want to have single-thread executors that make use of non-Send data?
• Am I correct that anyone building on tokio, or using a framework like tide, will basically want all futures to be sendable?
• Anything missing from my summary above? Any other options we may be overlooking?

It seems to me that the problem here is that for any given project you may care or not care about sendability, but you care kind of “uniformly”. i.e., it may be that only 15% of people using futures want a single-threaded executor, but for those that do they basically always want non-sendable futures, so if we have an opt-out like async(?Send), they’d have to write it all the time.

Details about the options

Details about the options

Bound. Either the foo bound needs some augmented form to specify that each of the future returned by p.process() is Send. We don’t have a clear syntax for naming that type right now, so I’ll write P::process:

fn foo<P: Process>(p: P) 
where P::process: Send // someway to specify that the async result is send
{
    let x = spawn(p.process()); // ERROR
}

Note that P: Send is neither correct nor sufficient – we care about the future that gets returned from p.process(), not p itself. (And process, in my example, is capturing a &P, so P would actually have to be Sync.)

Opt-in. Or we could have some sort of opt-in:

trait Process {
    async(Send) fn process(&self);
}

But it seems like quite possibly like the thing that people will want almost all the time.

Opt-out. We could say all async fn requires that futures are sendable. But in that case we would have async(?Send) fn for things that are not required to be sendable. Writing that a lot seems like an ergonomic hit.

Or we could have inconsistent defaults, though I think this would be really bad.

[josh]

I definitely think we shouldn’t make the defaults vary between trait methods and non-trait functions.

For the benefit of people like me who aren’t type theory experts, why do trait methods have this limitation, and non-trait functions don’t? Could we push on that a bit, and ask why that limitation exists and if we can do something about it?

[nikomatsakis]

@josh

For the benefit of people like me who aren’t type theory experts, why do trait methods have this limitation, and non-trait functions don’t? Could we push on that a bit, and ask why that limitation exists and if we can do something about it?

The answer is this:

If you call a function directly, you know precisely what future is being returned. Basically, we're leaking details about the fn body, in some sense -- or at least about the otherwise hidden type that it returns. This is a property of how impl trait works.

Essentially, when you write this:

fn foo() -> impl Future<..> { ... }

which is what an async fn desugars to, you are allowed to write a function that returns a non-send future or a send future. Callers of foo() cannot see exactly what type you returned, but they are able to tell if that type is Send or not.

In contrast, when you write code against a trait, you are being abstract over all implementors of the trait, so you can only assume the requirements that the trait states explicitly. So it might be that there are two impls, one of which returns a sendable future, and one of which doesn't.

[RustyYato]

Because trait methods are actually a family of methods who share a similar signature, and when you impose a rule on the trait method you impose that rule on every method in the family. But free functions or methods only descrobe themselves.

[nikomatsakis]

There’s another aspect to this, but I think it’s not the high-order bit. Still, an interesting wrinkle that @withoutboats raised.

If we did have a default of send, we can give better diagnostics. The problem is that now, you only detect that a future is sendable when you try to spawn a thread that is using it. This may be quite removed from the code that is preventing it from being sendable.

An example that @skade ran into (playground):

async fn foo() { 
  bar().await;
}
 
async fn bar() {
  let x = some_mutex.lock();
  baz().await;
}

async fn baz() { ... }

Here, the future returned by foo() is not send – this is because it embeds the future returned by bar(), and that future may have a mutex guard live across an await. Mutex guards are not sendable. Therefore, you get an error, because (in some states) the future may not be sendable across threads. But the error is fairly opaque, since it occurs at some point where we call foo(), but it has to do with the body of bar(). You can see it in the playground.

If we just required async fn bar() to return a send future in the first place, then we’d be able to give a more local error more easily.

Alternatively, though, we can probably overhaul the compiler and get super smart. =)

[nikomatsakis]

@nikomatsakis

It seems to me that the problem here is that for any given project you may care or not care about sendability, but you care kind of “uniformly”. i.e., it may be that only 15% of people using futures want a single-threaded executor, but for those that do they basically always want non-sendable futures, so if we have an opt-out like async(?Send) , they’d have to write it all the time .

Oh, I should add that @cramertj says that, in Fuschia, they use a ton of single-thread executors, and so they are very concerned about a "send by default" design. Meant to include that in the original post.

[dekellum]

It might be worth noting, possibly for use as a workaround with current impl, that when using what the async/await RFC calls the initialization pattern, you can add + Send to the return type where applicable.

Isn’t the initialization pattern, non-async fn with terminal async block, currently the only way to do this via trait methods?

[cramertj]

@nikomatsakis

Alternatively, though, we can probably overhaul the compiler and get super smart. =)

As a less tongue-in-cheek version of this, I really do think that the right solution here is that we should work on improving the diagnostics for these types of scenarios. It's not crazy to me to imagine this:

error[E0277]: `std::sync::MutexGuard<'_, u32>` cannot be sent between threads safely
  --> src/main.rs:23:5
   |
23 |     is_send(foo());
   |     ^^^^^^^ `std::sync::MutexGuard<'_, u32>` cannot be sent between threads safely
   |
   = help: within `impl std::future::Future`, the trait `std::marker::Send` is not implemented for `std::sync::MutexGuard<'_, u32>`
   = note: required because it appears within the type `for<'r, 's> {&'r std::sync::Mutex<u32>, std::sync::MutexGuard<'s, u32>, impl std::future::Future, ()}`
   = note: required because it appears within the type `[static generator@src/main.rs:13:30: 16:2 x:&std::sync::Mutex<u32> for<'r, 's> {&'r std::sync::Mutex<u32>, std::sync::MutexGuard<'s, u32>, impl std::future::Future, ()}]`
   = note: required because it appears within the type `std::future::GenFuture<[static generator@src/main.rs:13:30: 16:2 x:&std::sync::Mutex<u32> for<'r, 's> {&'r std::sync::Mutex<u32>, std::sync::MutexGuard<'s, u32>, impl std::future::Future, ()}]>`
   = note: required because it appears within the type `impl std::future::Future`
   = note: required because it appears within the type `impl std::future::Future`
   = note: required because it appears within the type `for<'r> {impl std::future::Future, ()}`
   = note: required because it appears within the type `[static generator@src/main.rs:9:16: 11:2 for<'r> {impl std::future::Future, ()}]`
   = note: required because it appears within the type `std::future::GenFuture<[static generator@src/main.rs:9:16: 11:2 for<'r> {impl std::future::Future, ()}]>`
   = note: required because it appears within the type `impl std::future::Future`
   = note: required because it appears within the type `impl std::future::Future`
note: required by `is_send`
  --> src/main.rs:5:1
   |
5  | fn is_send<T: Send>(t: T) {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^

becoming this:

error[E0277]: `foo()` cannot be sent between threads
note: the `is_send` function requires `T: Send`:
  --> src/main.rs:5:1
   |
5  | fn is_send<T: Send>(t: T) {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^
but in this invocation, the `foo()` is not `Send`
  --> src/main.rs:23:5
   |
23 |     is_send(foo());
   |             ^^^^^
because it is of type `impl Future<Output = ()>`
which contains an `std::sync::MutexGuard<'_, u32>`
note: pass `--full-error-type-info` for more information

It's certainly not trivial, but something like this would improve a ton of other existing scenarios around impl Trait. I don't personally think that async fn is particularly "special" here in comparison with other -> impl Trait functions. The error messages are similarly bad for -> impl Future / -> impl Iterator and this will be a problem again when we consider generators (gen fn).

I'd hazard a guess that requiring explicit Send opt-outs for generators is even more controversial than requiring them for Futures, yet I think it's important that we be consistent between the two.

[nikomatsakis]

@cramertj

As a less tongue-in-cheek version of this, I really do think that the right solution here is that we should work on improving the diagnostics for these types of scenarios.

In case it wasn't clear, I actually agree that it's plausible we can do that. This is partly why I said this issue seemed less crucial to me. It's just easier if we have a more local error, of course.

[yoshuawuyts]

Am I correct that anyone building on tokio, or using a framework like tide, will basically want all futures to be sendable?

That's accurate.

How often will people want to have single-thread executors that make use of non-Send data?

For non-embedded systems using single-threaded executors is generally done for performance reasons -- execution can be slightly sped up in some cases by not having to worry about threads.

But it's desirable to be able to swap between executors with ease, which in practice means all Futures have a Send bound. This is the same for WebAssembly targets too.

---

I can't comment on how this would interact with single-threaded non-std environments. @nemo157 might perhaps be able to share more on this.

[carllerche]

There are plenty of cases of Tokio being used with !Send futures. Tokio 100% supports that case and people use it. requiring all async fns to be Send is a non-starter IMO.

That said, Send is probably most common.

[Nemo157]

@nikomatsakis

If we stick with the current design

Which specific design are you referring to here? It's been a long time since I remember seeing a full sketch of how async fn in traits could work. Wow, searching back for it the last major thread on this is over a year old:

Pre-Pre-RFC: async methods & bounding async fns

Motivation The async fn RFC (in the process of being implemented) has two limitations: It only supports free async functions, not async trait methods (which allows async functions to play a role in our polymorphism system). It does not provide a way to require that your async fn’s future implements a trait like Send. This can often be very important. These two problems were sort of related. This pre-RFC is a sketch of how they could be solved. The main insight is that we can take inspiration…

There are (hopefully) soon to be implemented features (GATs + existential types) that will allow at least basic usage of async in traits, these don't currently encounter the same issue because you can choose whether to add the Send bound to the associated type:

trait Process {
  type ProcessFuture<'a>: Future<Output = ()> + Send + 'a;

  fn process(&self) -> Self::ProcessFuture<'_>;
}

impl Process for () {
  type ProcessFuture<'a> = impl Future<Output = ()> + Send + 'a;

  fn process(&self) -> Self::ProcessFuture<'_> {
    async { println!("processing"); }
  }
}

fn foo<P: Process>(p: P) {
    let x = spawn(p.process());
}

The issue arises only once a new feature is added to allow the async fn syntax in traits (whether it's direct sugar for the pattern above, or something different but similar).

[rtsuk]

I’ll second @cramertj here. I’m working on a Fuchsia-specific user-interface framework in Rust and it is strictly single-threaded. Having to add concurrency primitives just to use futures would be unappealing.

[betamos]

(Referred here by Taylor, I’m also on Fuchsia)

The biggest source of confusion for me was that Send is implicit, causing deferred non-local errors. Me and several others ran into the issue where a MutexGuard was used across an await point. Many of us couldn’t figure out why and needed to ask for help.

I like to see these errors before I actually schedule the methods on a multi-threaded executor. If I write a library, I want compiler errors even if the method is never used. Tagging/annotating a block of code, such as a file, type, trait or similar in one place as being Send (or not Send), would be helpful. (I don’t know enough about Rust to suggest any feasible such unit).

In a perfect world, the default wouldn’t be so terribly important, because it would be simple to change from one to another across a code base. For instance, going from a single-threaded to a multi-threaded executor is a valid use-case as a piece of code needs to be more performant.

[newpavlov]

I also find it really strange that this code:

fn foo() -> impl Foo { .. }

fn bar(f: impl Foo + Send) { .. } 

bar(foo())

Compiles if foo returns Send type and fails with compilation error otherwise. I believe such leakage of an inner implementation detail goes against Rust principles and we should use explicit impl Foo + Send in situations like this. Was this an intentional design? And if so was it discussed and what is rationale behind it?

As for async, I think we need a way to add additional trait bounds to return type, one way to do it is sync(Send + Foo) foo() -> T { .. } (unfortunately it’s a bit ugly… alternatively we could use attributes), which will be translated to fn foo() -> impl Future<T> + Send + Foo.

[cramertj]

With respect to traits, there’s a somewhat “clever” thing that I can imagine “doing the right thing” in 95% of scenarios: a future returned from an async method (or gen fn method in the future) is Send IFF all the arguments to the method are Send and Sync (Sync because you want to also be able to hold references to the argument types within the Future), unless someone has manually annotated the async fn in the trait definition as Send or !Send (I don’t care particularly what the syntax for this is).

This will “do the right thing” in nearly all scenarios, excepting where a !Send type is acquired from “somewhere” outside of the arguments to the function. it’s never the case that an async fn with a !Send argument is Send (since at the very least the argument is held onto and dropped at the start of the Future). !Sync arguments are a bit trickier, but I’d imagine these would be extraordinarily rare in scenarios where the Future itself was expected to be Send.

WDYT?

[cramertj]

@newpavlov

I also find it really strange that this code:

fn foo() -> impl Foo { .. }

fn bar(f: impl Foo + Send) { .. } 

bar(foo())

Compiles if foo returns Send type and fails with compilation error otherwise. I believe such leakage of an inner implementation detail goes against Rust principles and we should use explicit impl Foo + Send in situations like this. Was this an intentional design? And if so was it discussed and what is rationale behind it?

Yes, this was always the design of impl Trait. Auto trait leakage was an intentional part of the design, and is necessary for many usecases where today it is impossible to represent a bound like impl SendIfAndOnlyIf<T: Send> explicitly.

[leodasvacas]

Don’t think that solves well the common case where &self is a parameter, such as in the first example. For async fn process(&self) to be Send we’d need to write the bound Self: Sync.

[cramertj]

In that case, the async fn future type would be Send if and only if Self: Send, which IMO is exactly what we want.

[anp]

I’ve used Futures and async fn in a few contexts, including the single-thread-focused Fuchsia codebase, writing Futures that perform thread-bound FFI code (i.e. a future which interacts with iOS Views on the main thread), and the more prevalent “do some server work and throw it in a work stealer”.

One question: does this same issue arise for generic async fns, where they have to specify the Send bound on their arguments if they want to be Send?

Another question: how does this differ from the need to annotate generic methods in traits with Send/Sync/Sized/'static bounds? That requirement is also inconvenient to type but is the status quo (perhaps relevant for consistency?).

If we model what async “desugars” to, does that give us a nice perspective for picking “Send required”/“Send not required” syntax?

async fn foo() -> String { ... }

// becomes roughly

fn foo() -> impl Future<Output = String> { ... }

// becomes roughly

fn foo<F>() -> F
where F: Future<Output = String>

// we can imagine adding *some* annotation which transitively turns this into

fn foo<F>() -> F
where F: Future<Output = String> + Send

Right? If this is the bound that conceptually we would emit from an annotation, is there a nice way to express this type bound in the source? Here’s a hasty idea:

async fn foo() -> String
where Send
{
    // ...
}