Would implicit `await` really be a bad idea?

[dpc]

While I was chatting on reddit about await syntax debate, I had a thought, that I wanted to write down, and maybe get shot down with some good arguments.

So, I’m doing node.js at dayjob, and await feels very redundant there. I mean … what else are you going to do? Writing async code in JS is just a constant busywork of awaiting everywhere for no benefit. Any time you have a Promise/Future you can either return it upwards, combine, or await. While in JS these might be sometimes ambiguous syntax-wise, in Rust with a strong type system I don’t think it would be a problem. It seems to me that in Rust await is mostly going to be here, compiler, I am aware that this can be a yieldpoint annotation busywork.

So the question is: are we (should we?) be really interested in a yieldpoints? I mean … you’ve already annotated the function/block as async one, right? So it is marked as “this code can yield”. From the logical perspective yield-points at await don’t really exist. It’s not like you can insert any code between future returning NotReady, and Yield upward that await generates. Even from a low-level/system-level code - having yield points explicitly marked does not seem that useful. Awaiting on a Future is just a blocking IO!

I’m aware that there is eg. a risk of holding a Mutex locked over a yieldpoint. But is it that any different than holding it over any other IO? We don’t annotate reading from a socket with some block keyword and we’re doing just fine, no? So what’s so different about awaiting on a Future, which logically is just blocking on a IO?

So my thesis is - Because Rust is already so explicit and strongly typed, there’s no need for await syntax and Futures could auto-magically Derefas await or something. That would kill a lot of redundant noise, without much downsides. It would also make converting code to async much easier, language easier to learn etc. Rust type-system guarantees that it’s not going to be a source of silly bugs, and “holding resources over a yieldpoint” is no different than “holding a resource over a blocking IO” which we already deal with just fine, and which can be a clippy lint or something of the same type (“locking over io”).

[scottmcm]

One doesn’t always await everything immediately, though. I don’t know what delaying an await looks like in the implicit world.

[yasammez]

Especially in the JS world I often find myself in the situation where I create half a dozen futures sequentially and then only

await Promise.all(future1, future2, future3);

How would that look in the implicit world?

[CAD97]

This is explicit async, implicit await. Kotlin does their suspending functions this way.

Personally, I really like this style, because it accomplishes making async code look sync moreso than “implicit async, explicit await”, and makes the lazy versus eager-to-first-await versus started-in-background dimension of futures a non-issue.

If you want to delay the execution of async fn do_work(), you put it in a closure, just the same as if you were doing this synchronously: async || { do_work() }. If you want to execute it now (from the perspective of the execution context), you just call it: do_work().

But this doesn’t work for Rust. Why? Because we want the usage of async fn() -> T and fn() -> impl Future<T> to be the same.

In Kotlin and any “explicit async, implicit await” system, -> impl Future turns the equation around, making the async explicit (returning a closure, effectively), and requires a async fn await method call in an async context to run it.

[CAD97]

Especially in the JS world I often find myself in the situation where I create half a dozen futures sequentially and then only

await Promise.all(future1, future2, future3);

How would that look in the implicit world?

async fn context() {
    let [ res1, res2, res3 ] = Async::all([
        async || task1(),
        async || task2(),
        async || task3(),
    ]);
}

(This API requires const generics but could be made similar to select! I suppose)

[H2CO3]

It has all the usual problems with anything implicit around expressions: what do you do when you don’t want it? If awaiting is implicit, there’s no way to tell apart awaiting from not awaiting.

(Of course that is, unless you try to hack around it by inverting the behavior altogether and requiring a “noawait” or whatever keyword, which now makes the entire thing context dependent because you would need to remove it if you moved the code outside an async context, making matters hacky and inconsistent.)

Another problem is very similar but not identical: what about expressions that aren’t async at all? Simple, synchronous function calls, variable, initialization, etc. How does one tell the compiler “don’t try to await this because it’s not going to work”? (No, checking if it impls Future is neither robust nor desirable because then we are back on square 1 as described by the previous point.)

So yes, definitely a bad idea IMO.

[matklad]

So yes, definitely a bad idea IMO.

That’s a too confident judgement, imo Kotlin works which is an empirical evidence that idea is not bad. It might not be applicable in Rust, but arguing that requires a more nuanced approach.

[dpc]

Depending on how exactly we do (semi-) implicit await, we figure out a method. My point is: I rather have a call/method /syntax to inhibit it 1% of the time, than keep typing explicit await 99% of the time without any befits.

[rpjohnst]

what do you do when you don’t want it? If awaiting is implicit, there’s no way to tell apart awaiting from not awaiting.

This is a silly argument- that question has already been answered in this thread, without a “noawait” keyword or any context dependence. It’s very simple: expressions which are calls to async fns are awaited; everything else is not, and to delay those calls you put them in async { .. } blocks, which we already have anyway.

But this doesn’t work for Rust. Why? Because we want the usage of async fn() -> T and fn() -> impl Future<T> to be the same.

This is something that some people want, but it’s not set it stone that we need it. With implicit await, the Future trait could have an async fn await() method, and anything that needs to return an impl Future would just get an extra .await() call. We lose seamless conversion between the two, but many cases where you need to convert will be due to changes in API anyway!

Alternatively, we could try to solve the problem that requires so much -> impl Future in the first place. The way arguments are captured and the resulting double-indentation of an async block is a pain regardless of which await syntax we choose, and fixing that would also reduce the need for seamless conversion between async fn and -> impl Future.

[vorner]

I don’t like the idea of implicit await because of readability. I find Rust very easy to read even if I don’t know the code, mostly because it contains a whole lot of hints. That’s not just sensible names, but all these & and mut at the caller side of functions, ? that may return and hints that the function being called might fail, etc ‒ basically, all the „line noise“ helps me to read Rust code.

In a similar way, explicit await would be another hint that would help me read and understand the code ‒ the fact that I can exit the function, unwind the stack all the way up to some unknown executor, arbitrary code happening in there in the same thread and then return is IMO quite important thing at least in certain kinds of applications.

So while I believe it could be possible to find some way to make the implicit thing work in Rust, I’d prefer not to, as I consider readability more important than writing shorter code.

And, by the way, I believe it makes sense to support both the old-style functional Futures + combinators and the new async syntax. But then, how does these thing interact? With explicit async I can kind of imagine what happens, but how do these two world work together with implicit one?

[dpc]

There’s no readability benefits. It’s all moot, I argue. At best as useful as annotating every blocking io, which we don’t do. The point is of async is to hide the details of generators. I’d like someone provide a practical example of how is it useful.

[josh]

I don’t consider explicit await “busywork”, any more than I consider explicit threading to be busywork. Frameworks like rayon make threads as convenient as possible, but I’d never want to have implicit thread spawning, for instance.

You don’t always want to await a future; sometimes you want to apply a combinator to combine it with another future, or return it as a future, or do any number of things other than await it.

[vorner]

I understand readability is a subjective thing. You may not need that information, but in the code I write, I often prefer to have it at hand.

Blocking is not the same thing as interleaving with arbitrary unknown code. If you call ordinary method, you kind of know what code is hidden inside, with async call you do not. And the „protective“ barrier of your thread does not apply there sometimes ‒ someone might touch „my“ thread locals, someone might access the RefCell I’m having locked. I better see that in the code on the first glance ‒ I don’t want to keep a RefCell locked across await, because that’s a path to get various panics that exhibit randomly, depending what gets scheduled inside my awaiting. I however can quite safely call blocking write with a locked RefCell.

I understand that for people coming from eg. python or node.js, the await thing might look like unnecessary stuff ‒ but these languages in general try to hide everything under the hood to make life of programmers easier. There are situations where the other approach ‒ give the programmer all the information ‒ might be reasonable. I believe Rust should aim for the latter.

[matklad]

I don’t consider explicit await “busywork”, any more than I consider explicit threading to be busywork. Frameworks like rayon make threads as convenient as possible, but I’d never want to have implicit thread spawning, for instance.

That’s an interesting analogy which I think actually argues for the opposite point. With threads, suspend points are implicit and may happen anywhere, and that causes no problems in Rust due to strict aliasing control.

Spawning separate threads of control is explicit in blocking Rust and in both flavors of the async. To give an example in Kotlin,

fun main() = runBlocking<Unit> {
//sampleStart
    val time = measureTimeMillis {
        val one = async { doSomethingUsefulOne() }  // explicit spawn!
        val two = async { doSomethingUsefulTwo() }
        println("The answer is ${one.await() + two.await()}")
    }
    println("Completed in $time ms")
//sampleEnd    
}

FWIW, in implicit await it’s even more explicit, because you need an async keyword for that.

[H2CO3]

“You can implement it in a programming language” doesn’t automatically mean that it’s a good idea. There are many things that are possible but not a good idea in general. (Also who said that I have to like the way Kotlin works or that it should work the same way in Rust? These two are very different languages.)

As for the accusation of not being nuanced: you can see that I’ve considered several aspects of it – you are free to disagree but that doesn’t make opposing opinions any less nuanced. Please stop using that as a counterargument, it’s not productive, just annoying.

[dpc]

Can you even share RefCell between async functions? Between generators probably yes, but what about multithread executors? Anyway it’s not much different than holding a mutex over blocking io call.

[vorner]

That async function won’t be Sync/Send, so I can only spawn it onto a single threaded executor.

And it’s not the same. With a mutex locked around a blocking io call, you get a terrible performance, because you block it longer than necessary. But others will just wait their turn with a mutex, RefCell will panic instead. With a mutex around an await, you’re more likely to get a deadlock than just terrible performance.

[dpc]

You can get a deadlock with the mutex over io, no problem. The fact that it isn’t a panic makes it even worse of an issue. In both cases clippy ought to come to the rescue. And keeping mutex/RefCell borrows tight is advised for this to not happen.

Not a big enough worry to justify sparkling all Rust code with a new keyword IMO. And that what happens with explicit await eg in Node. Await, await, await, await, await…

[rpjohnst]

This problem happens in all sorts of configurations, with or without explicit await. The solution is to use the right lock type- Mutex for OS threads, and an async-aware equivalent for futures, and explicit await doesn’t help at all with that decision.

[Nemo157]

One thing I don’t remember seeing brought up in any of the implicit await discussions yet is auto-traits like Send. With explicit await you have an explicit point at which you can ensure a !Send variable is dead so that it won’t affect the generated Future, with implicit await you would have to know which function calls are async when you’re ensuring a !Send variable is dead across yield points.

For example (assuming #57478 is fixed, otherwise needs to use braced scopes):

async {
    let foo: NotSend = bar();
    baz();
    drop(foo);
    quux();
}

can you re-order the call to quux() before the drop without affecting the Sendness of the future? You can’t tell locally here, you would need to look at the definition of quux to know.