How often do you want non-send futures?

So I recently became aware of something that I had not considered regarding async fn – in particular, the current default where we don’t require the resulting future to be Send doesn’t work out so well once we scale up to async fn in traits. (I take it many people are aware of this for some time, but I at least hadn’t thought about it before and I’d like to talk it out.)

The problem

The current async fn definition returns an impl Future – it does not promise that the resulting future is send. In practice, this is fine, because auto traits will automatically figure out whether it should be send or not. Very ergonomic, very nice:

async fn process() { }

fn foo() {
  spawn(process()); // requires a sendable future; works b/c auto-traits
}

However, there is a wrinkle. When we support async fn in traits, things won’t work out as well:

trait Process {
  async fn process(&self);
}

fn foo<P: Process>(p: P) {
    let x = spawn(p.process()); // ERROR
}

The problem here is that the definition of foo must be valid for all processes Process types, including those whose resulting future is not send.

Options

If we stick with the current design, we can resolve this in three ways (detailed below):

  • Some form of bound that foo can use to specify that P::process returns a Send future
  • Some way to opt-in to sendability async(Send) fn process()
  • A different default in traits (e.g., async fns in traits are send by default)

Alternatively, if we change the design we could:

  • default to sendable futures, but offer an opt-out like async(?Send).

None of these options look that appealing to me right now. They all seem to impose a cost somewhere.

Questions

I have a few questions I would like feedback on:

  • How often will people want to have single-thread executors that make use of non-Send data?
  • Am I correct that anyone building on tokio, or using a framework like tide, will basically want all futures to be sendable?
  • Anything missing from my summary above? Any other options we may be overlooking?

It seems to me that the problem here is that for any given project you may care or not care about sendability, but you care kind of “uniformly”. i.e., it may be that only 15% of people using futures want a single-threaded executor, but for those that do they basically always want non-sendable futures, so if we have an opt-out like async(?Send), they’d have to write it all the time.

Details about the options

Details about the options

Bound. Either the foo bound needs some augmented form to specify that each of the future returned by p.process() is Send. We don’t have a clear syntax for naming that type right now, so I’ll write P::process:

fn foo<P: Process>(p: P) 
where P::process: Send // someway to specify that the async result is send
{
    let x = spawn(p.process()); // ERROR
}

Note that P: Send is neither correct nor sufficient – we care about the future that gets returned from p.process(), not p itself. (And process, in my example, is capturing a &P, so P would actually have to be Sync.)

Opt-in. Or we could have some sort of opt-in:

trait Process {
    async(Send) fn process(&self);
}

But it seems like quite possibly like the thing that people will want almost all the time.

Opt-out. We could say all async fn requires that futures are sendable. But in that case we would have async(?Send) fn for things that are not required to be sendable. Writing that a lot seems like an ergonomic hit.

Or we could have inconsistent defaults, though I think this would be really bad.

11 Likes

I definitely think we shouldn’t make the defaults vary between trait methods and non-trait functions.

For the benefit of people like me who aren’t type theory experts, why do trait methods have this limitation, and non-trait functions don’t? Could we push on that a bit, and ask why that limitation exists and if we can do something about it?

1 Like

The answer is this:

If you call a function directly, you know precisely what future is being returned. Basically, we’re leaking details about the fn body, in some sense – or at least about the otherwise hidden type that it returns. This is a property of how impl trait works.

Essentially, when you write this:

fn foo() -> impl Future<..> { ... }

which is what an async fn desugars to, you are allowed to write a function that returns a non-send future or a send future. Callers of foo() cannot see exactly what type you returned, but they are able to tell if that type is Send or not.

In contrast, when you write code against a trait, you are being abstract over all implementors of the trait, so you can only assume the requirements that the trait states explicitly. So it might be that there are two impls, one of which returns a sendable future, and one of which doesn’t.

1 Like

Because trait methods are actually a family of methods who share a similar signature, and when you impose a rule on the trait method you impose that rule on every method in the family. But free functions or methods only descrobe themselves.

2 Likes

There’s another aspect to this, but I think it’s not the high-order bit. Still, an interesting wrinkle that @withoutboats raised.

If we did have a default of send, we can give better diagnostics. The problem is that now, you only detect that a future is sendable when you try to spawn a thread that is using it. This may be quite removed from the code that is preventing it from being sendable.

An example that @skade ran into (playground):

async fn foo() { 
  bar().await;
}
 
async fn bar() {
  let x = some_mutex.lock();
  baz().await;
}

async fn baz() { ... }

Here, the future returned by foo() is not send – this is because it embeds the future returned by bar(), and that future may have a mutex guard live across an await. Mutex guards are not sendable. Therefore, you get an error, because (in some states) the future may not be sendable across threads. But the error is fairly opaque, since it occurs at some point where we call foo(), but it has to do with the body of bar(). You can see it in the playground.

If we just required async fn bar() to return a send future in the first place, then we’d be able to give a more local error more easily.

Alternatively, though, we can probably overhaul the compiler and get super smart. =)

1 Like

Oh, I should add that @cramertj says that, in Fuschia, they use a ton of single-thread executors, and so they are very concerned about a “send by default” design. Meant to include that in the original post.

10 Likes

It might be worth noting, possibly for use as a workaround with current impl, that when using what the async/await RFC calls the initialization pattern, you can add + Send to the return type where applicable.

Isn’t the initialization pattern, non-async fn with terminal async block, currently the only way to do this via trait methods?

2 Likes

As a less tongue-in-cheek version of this, I really do think that the right solution here is that we should work on improving the diagnostics for these types of scenarios. It’s not crazy to me to imagine this:

error[E0277]: `std::sync::MutexGuard<'_, u32>` cannot be sent between threads safely
  --> src/main.rs:23:5
   |
23 |     is_send(foo());
   |     ^^^^^^^ `std::sync::MutexGuard<'_, u32>` cannot be sent between threads safely
   |
   = help: within `impl std::future::Future`, the trait `std::marker::Send` is not implemented for `std::sync::MutexGuard<'_, u32>`
   = note: required because it appears within the type `for<'r, 's> {&'r std::sync::Mutex<u32>, std::sync::MutexGuard<'s, u32>, impl std::future::Future, ()}`
   = note: required because it appears within the type `[static generator@src/main.rs:13:30: 16:2 x:&std::sync::Mutex<u32> for<'r, 's> {&'r std::sync::Mutex<u32>, std::sync::MutexGuard<'s, u32>, impl std::future::Future, ()}]`
   = note: required because it appears within the type `std::future::GenFuture<[static generator@src/main.rs:13:30: 16:2 x:&std::sync::Mutex<u32> for<'r, 's> {&'r std::sync::Mutex<u32>, std::sync::MutexGuard<'s, u32>, impl std::future::Future, ()}]>`
   = note: required because it appears within the type `impl std::future::Future`
   = note: required because it appears within the type `impl std::future::Future`
   = note: required because it appears within the type `for<'r> {impl std::future::Future, ()}`
   = note: required because it appears within the type `[static generator@src/main.rs:9:16: 11:2 for<'r> {impl std::future::Future, ()}]`
   = note: required because it appears within the type `std::future::GenFuture<[static generator@src/main.rs:9:16: 11:2 for<'r> {impl std::future::Future, ()}]>`
   = note: required because it appears within the type `impl std::future::Future`
   = note: required because it appears within the type `impl std::future::Future`
note: required by `is_send`
  --> src/main.rs:5:1
   |
5  | fn is_send<T: Send>(t: T) {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^

becoming this:

error[E0277]: `foo()` cannot be sent between threads
note: the `is_send` function requires `T: Send`:
  --> src/main.rs:5:1
   |
5  | fn is_send<T: Send>(t: T) {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^
but in this invocation, the `foo()` is not `Send`
  --> src/main.rs:23:5
   |
23 |     is_send(foo());
   |             ^^^^^
because it is of type `impl Future<Output = ()>`
which contains an `std::sync::MutexGuard<'_, u32>`
note: pass `--full-error-type-info` for more information

It’s certainly not trivial, but something like this would improve a ton of other existing scenarios around impl Trait. I don’t personally think that async fn is particularly “special” here in comparison with other -> impl Trait functions. The error messages are similarly bad for -> impl Future / -> impl Iterator and this will be a problem again when we consider generators (gen fn).

I’d hazard a guess that requiring explicit Send opt-outs for generators is even more controversial than requiring them for Futures, yet I think it’s important that we be consistent between the two.

6 Likes

In case it wasn’t clear, I actually agree that it’s plausible we can do that. This is partly why I said this issue seemed less crucial to me. It’s just easier if we have a more local error, of course.

1 Like

Am I correct that anyone building on tokio, or using a framework like tide, will basically want all futures to be sendable?

That’s accurate.

How often will people want to have single-thread executors that make use of non-Send data?

For non-embedded systems using single-threaded executors is generally done for performance reasons – execution can be slightly sped up in some cases by not having to worry about threads.

But it’s desirable to be able to swap between executors with ease, which in practice means all Futures have a Send bound. This is the same for WebAssembly targets too.


I can’t comment on how this would interact with single-threaded non-std environments. @nemo157 might perhaps be able to share more on this.

3 Likes

My take is that futures returned by async fn in traits should be Send by default, with the ability to opt-out like this:

trait Trait {
    // Case 1: The future is `?Send`.
    fn foo(&self) -> impl Future<Output = ()>;

    // Case 2: The future is `Send`.
    async fn foo(&self);

    // Case 3: The future is `Send`.
    // Same thing as case 2, except written in a longer form.
    fn foo(&self) -> impl Future<Output = ()> + Send;
}

And here’s why. Consider this:

impl Struct {
    // The returned future borrows `arg`.
    async fn foo(&self, arg: &i32);

    // The returned future only borrows `self` and not `arg`.
    fn foo<'a>(&self, arg: &i32) -> impl Future<Output = ()> + 'a;
}

Async functions outside traits already have strong assumptions about borrowing by default (they borrow all arguments), and the only way to opt-out from that is to write the async fn in the longer fn -> impl Future form.

So it makes sense to me to likewise default to strong assumptions in trait methods when it comes to Send, with the ability to opt-out in the same manner by spelling out the signature in the longer form.

2 Likes

There are plenty of cases of Tokio being used with !Send futures. Tokio 100% supports that case and people use it. requiring all async fns to be Send is a non-starter IMO.

That said, Send is probably most common.

10 Likes

Which specific design are you referring to here? It’s been a long time since I remember seeing a full sketch of how async fn in traits could work. Wow, searching back for it the last major thread on this is over a year old:

There are (hopefully) soon to be implemented features (GATs + existential types) that will allow at least basic usage of async in traits, these don’t currently encounter the same issue because you can choose whether to add the Send bound to the associated type:

trait Process {
  type ProcessFuture<'a>: Future<Output = ()> + Send + 'a;

  fn process(&self) -> Self::ProcessFuture<'_>;
}

impl Process for () {
  type ProcessFuture<'a> = impl Future<Output = ()> + Send + 'a;

  fn process(&self) -> Self::ProcessFuture<'_> {
    async { println!("processing"); }
  }
}

fn foo<P: Process>(p: P) {
    let x = spawn(p.process());
}

The issue arises only once a new feature is added to allow the async fn syntax in traits (whether it’s direct sugar for the pattern above, or something different but similar).

I’ll second @cramertj here. I’m working on a Fuchsia-specific user-interface framework in Rust and it is strictly single-threaded. Having to add concurrency primitives just to use futures would be unappealing.

7 Likes

(Referred here by Taylor, I’m also on Fuchsia)

The biggest source of confusion for me was that Send is implicit, causing deferred non-local errors. Me and several others ran into the issue where a MutexGuard was used across an await point. Many of us couldn’t figure out why and needed to ask for help.

I like to see these errors before I actually schedule the methods on a multi-threaded executor. If I write a library, I want compiler errors even if the method is never used. Tagging/annotating a block of code, such as a file, type, trait or similar in one place as being Send (or not Send), would be helpful. (I don’t know enough about Rust to suggest any feasible such unit).

In a perfect world, the default wouldn’t be so terribly important, because it would be simple to change from one to another across a code base. For instance, going from a single-threaded to a multi-threaded executor is a valid use-case as a piece of code needs to be more performant.

12 Likes

I also find it really strange that this code:

fn foo() -> impl Foo { .. }

fn bar(f: impl Foo + Send) { .. } 

bar(foo())

Compiles if foo returns Send type and fails with compilation error otherwise. I believe such leakage of an inner implementation detail goes against Rust principles and we should use explicit impl Foo + Send in situations like this. Was this an intentional design? And if so was it discussed and what is rationale behind it?

As for async, I think we need a way to add additional trait bounds to return type, one way to do it is sync(Send + Foo) foo() -> T { .. } (unfortunately it’s a bit ugly… alternatively we could use attributes), which will be translated to fn foo() -> impl Future<T> + Send + Foo.

11 Likes

With respect to traits, there’s a somewhat “clever” thing that I can imagine “doing the right thing” in 95% of scenarios: a future returned from an async method (or gen fn method in the future) is Send IFF all the arguments to the method are Send and Sync (Sync because you want to also be able to hold references to the argument types within the Future), unless someone has manually annotated the async fn in the trait definition as Send or !Send (I don’t care particularly what the syntax for this is).

This will “do the right thing” in nearly all scenarios, excepting where a !Send type is acquired from “somewhere” outside of the arguments to the function. it’s never the case that an async fn with a !Send argument is Send (since at the very least the argument is held onto and dropped at the start of the Future). !Sync arguments are a bit trickier, but I’d imagine these would be extraordinarily rare in scenarios where the Future itself was expected to be Send.

WDYT?

5 Likes

Yes, this was always the design of impl Trait. Auto trait leakage was an intentional part of the design, and is necessary for many usecases where today it is impossible to represent a bound like impl SendIfAndOnlyIf<T: Send> explicitly.

3 Likes

Don’t think that solves well the common case where &self is a parameter, such as in the first example. For async fn process(&self) to be Send we’d need to write the bound Self: Sync.

In that case, the async fn future type would be Send if and only if Self: Send, which IMO is exactly what we want.

1 Like