How often do you want non-send futures?

It would break fuchsia, our largest nightly user, not to have any support for non-Send futures in the MVP. Seems untenable to me.

Hm, I don’t quite understand. Is this the only thing blocking fuchsia of getting off or would it mean high churn if the syntax changes? In that case, I definitely agree.

What is special about Send here? What about Sync? Since Sized came up, what about !Sized? (Well, I realize return values can’t be !Sized, so I suppose that’s out of the question anyway.)

Perhaps there should be a more general "trait bounds for async" syntax? Something like…

async<B> foo() -> Result<Bar, Baz>
    where B: Send;

(I realize this ups the noise factor for whichever bound, Send or ?Send, is not the default, and I’m not going to weigh in on what the default should be…)

4 Likes

Arguably, with good enough error messages and a simple enough fix, breaking some copy-and-paste code might be a good thing.

I know that Fuschia would very much like to make it possible to write async fn that do not require nightly because that enables them to have more of a “Rust ecosystem” – @cramertj can confirm, but how I understand it is that Rust crates that use nightly are confined to be “in tree” so that they can be kept up to date as nightly evolves. But I don’t really know a lot more than that, so further details would have to come from @cramertj or other Fuschia folks. :slight_smile:

1 Like

I think it's a reasonable question whether there are other traits that would want to be the default -- but there are some special things about Send.

  • We know that some futures need to be send but others don't -- and imposing Send involves limiting the sorts of things the future can do (i.e., it can't access a shared Cell or RefCell, use an Rc, etc).
  • The Sync trait says that something can be safely shared across threads (as opposed to transferred from thread to thread). But sharing a future across threads doesn't make sense: the poll method on Future is &mut self, so you couldn't do anything with a shared future.
  • Other traits like Debug would probably be useful, but as @withoutboats noted elsewhere, we would ideally implement these uniformly for all async fns, and there isn't really a downside to implementing them (but doing this well would require some form of specialization, I imagine, so that we can have a fallback).
4 Likes

I have an executor wrapping the non-blocking APIs of the MPI library that, depending on how the MPI library was compiled or initialized, can only be polled from the exact same thread the first poll happened.

Before the first poll, I can send the future across threads without issues. I’m not sure if this means my futures need to be !Send ? Right now, I just Pin my futures before starting to poll them, and then the Pin guarantees that they won’t be moved across threads until they are dropped or the Future completes. Is that correct?

AFAICT, whether my Futures are Send or not will depend on which data they contain, e.g., if the user stores a reference to something that isn’t Send inside a future, then the Future will be automatically !Send.

Often (depending on compilation options/initialization), the MPI executor needs to live in the main thread, and applications using them are also often single threaded. That is, the executor is not polled continuously in a separate thread, but manually told to poll all futures “once” (until they can’t make more progress) at specific points within the main thread, and then the executor yields back to the main thread, so that the thread can do some more work, until the executor needs to be polled again (at some point there is a blocking poll). That is, storing !Send data-types inside a Future isn’t an issue there, and something that applications might reasonably want to do.

1 Like

Pin doesn't stop moving the future across threads, you can pass a Pin<Box<Foo>> across threads and poll it on multiple of them if Foo: Send (playground).

I guess that means I need to make all my Futures !Send, at least depending on some feature flag.

1 Like

This is correct. The Send auto trait assumes that things can be sent across threads if they have no access to thread-local data. (This is true for safe code, although it doesn't capture the effects of e.g. accessing thread-locals, which can cause surprising results -- but not unsafety.)

I found this comment incomprehensible but I think we are actually agreeing?

I don’t think we are. If I understood you correctly, I believe you suggested that the parser should reject:

async(Foo + Bar) fn myfun<'a>() {}

and only accept:

async(?Send) fn myfun() {}

I am suggesting that if we go down this route, we should syntactically and semantically accept:

async(Foo + Send) // modulo ?Send / Send discussion
fn bar() -> R {}

which is desugared into:

fn bar() -> impl Future<Output = R> + Foo + Send {}

via lowering, because it would be ad-hoc not to do so and also inconsistent with how ?Sized behaves.

The syntactic / semantic distinction is also relevant because Rust’s surface syntax alone has stability guarantees even if the syntax later results in semantic errors. For example:

#[cfg(FALSE)] struct A { f: impl Copy }

will compile but:

struct A { f: impl Copy }

will not, and this has stability implications.

3 Likes

I presume that B is a type in this context, but what does it refer to, the outer return type?

I think @withoutboats was just saying that initially we could be conservative but eventually aim to accept arbitrary bounds, and not to specifically say that we should reject async(?Send) in the parser (as opposed to later in lowering).

but to what end?

From my perspective, this conservatism is a complication both in terms of the grammar, time investment, and compiler complexity.

I also don’t see much reason to be conservative. (But I doubt it was the “high-order bit” for @withoutboats either – but I should let them speak for themselves.)

1 Like

No I'm saying what you're saying (but I think in the short term we should hard reject bounds other than ?Send after parsing. This is because some have more complicated implications, Unpin in particular).

2 Likes

Yes, since there’s no other way to specify any information about it. But conceptually, it should only be used to specify traits that must be satisfied by all captured state.

I like where this goes in terms of reuse of existing syntax. Possible alternative: the async keyword is also available as the generic variable directly?

async foo() -> Result<Bar, Baz>
    where async: Send;
7 Likes

In that case, I do think we should consider these complicated implications before introducing async(Send) or similar so that we can tell whether we will end up with something uniform and coherent in the end. It would be concerning would we not.

Intriguing; I concur with @dekellum that this appears to be good for reuse and I think it's conceptually simple. I wonder however whether this has any use beyond auto traits and lifetimes since the compiler generates the actual return type itself. If we had some sort of -XScopedTypeVariables for this existential specifically, I suppose you could allow:

async<R> fn foo() -> u8 where R: MyTrait {
    impl MyTrait for R {
        ...
    }

    0
}
3 Likes