Would implicit `await` really be a bad idea?

Or better (maybe worse for compiler writers),

Inside an async code block, if a Future being awaited will make it type check, then do so. Not awaiting only on otherwise. In other words, if awaiting will fail the type check, try again without awaiting.

I assume this will significantly increase the compile cost though, specifically the cases can quickly increases exponentially.

It would also be a bit confusing to users I guess. Some types would somehow get active by just letting them lying on the floor. Eg. if you have fn x() -> Vec<i32> and do x();, then it is just dropped and nothing happens. If you have fn y() -> SomeFuture and do y();, it doesn’t get dropped, it starts doing stuff. Why some types just die and some types act?

That's not quite correct: something did happen! In particular, the call to x() happened (which involved some computation).

Similarly, when you call y(), stuff also happens, the only difference is this time it's asynchronous computation.

So really the only difference between x() and y() is whether the computation is synchronous or asynchronous.

No, with x, the computation happens inside x. With y, something also happens outside of y. Inside y, the future is constructed. It needs to be polled after y terminated.

I think this boils down to these questions:

  • Does an async function return a future, or is it „magical” form of function? I believe the current design is it returns a function.
  • Is a future a value like anything else (like vectors, like errors, like iterators), or something completely different? Again, to me, it seems the current design is that a future is a value.
  • So, to make the auto-await work, you need something to happen on the caller side of the function call. But then, how is that consistent with other ways of calling stuff?

Does that sound like technical details? Maybe it is, but in Rust I think technical details are not usually papered over.

Those are all good points. However:

  1. Rust already has some implicitness, e.g. IO is not represented in the type (unlike some other languages like Haskell).

  2. The fact that the automatic await only happens inside of async means that it is explicit, maybe just not as explicit as some would like.

So you are right that it comes down to how “built-in” should Futures be? Should they just be a completely normal part of the language like function calls and IO?

Or should they be treated as a regular Rust type, and not magical at all?

And would the answer be different if Futures had always been a part of Rust, rather than added later?

I would even argue that awaiting immediately should be avoided wherever possible, and should not be the normal or default operation. Awaiting futures immediately can a recipe for maximizing latency.

The first futures API that I used was an internal library at Amazon back in 2005. The company had developed many best practices for minimizing latency. One of the rules was to schedule futures as early as possible, and await them as late as possible. This means that typically there is a lot of space between where a future is created and where it is resolved.

For example, we would NOT do this. This would cause the parent task to block on the child task before starting its next chunk of work:

let x = await async_work(); // bad
let y = do_some_stuff();
do_more_stuff(x, y);

And we would NOT do this, which requires the parent task to complete the first chunk of work before scheduling the child task (which will then block the parent).

let y = do_some_stuff();
do_more_stuff(await async_work(), y); // bad

The correct way to write the code is this, which allows the child task and parent task to run concurrently up until the point where the child task's result is needed:

let x = async_work();
let y = do_some_stuff();
do_more_stuff(await x, y); // good

(These optimizations get more complex when a parent task spawns several child tasks, some of which depend on each other, but the principles remain the same. Typically you end up with a bunch of async calls near the start of a function, and bunch of awaits near the end.)

I know these practices are not yet universal, and many programmers do tend to await futures immediately. However, as async programming matures, I think that more people will rediscover these rules. We should not adopt any language design that makes the "good" code above harder to read/write than the "bad" code.

20 Likes

This makes perfect sense in a language like JavaScript where Promises run immediately.

But in Rust Futures are delayed until first polled, so it's much much harder to preemptively run a Future like that.

Instead, it's better to make the parallelism more explicit by using join! or similar.

Do you have any ideas about how to do preemptive work in Rust (aside from join!)?

2 Likes

This all applies equally well to Rust futures. Even in the single-threaded case, if the child task is performing IO then the parent task can continue to do work after spawning the async IO task, and delay blocking until the IO result is needed. The executor running the parent task will poll the child task at the point where the parent task awaits it.

No, it doesn't. This is a rather subtle but major difference between how futures in Rust work to C#/JS (and probably others, but those are the ones I am familiar with). Rust futures do not run in parallel to synchronous work that is being performed within the same task and cannot be expected to make any progress until awaited.

If async_work() spawns off a separate task on the executor and just returns a channel that will be provided with the result of that task, then yes it will run in parallel; but at that point you lose a lot of what makes Rust futures unique, instead of having a highly optimizable state machine you are basically back to the overhead of having a lot of dynamic tasks calling continuations when complete.

5 Likes

Rust futures may not run code in parallel with synchronous work in the same task (except in cases as you note where threads are involved). But they can allow the OS, for example, to perform other work concurrently with the parent task.

A concrete example: If the async work function above is tokio::net::TcpStream::connect, then at the point where it is called, it immediately makes a connect syscall.† It doesn’t block on the result of the call, but returns a future that can be polled for the result. However, the libc::connect call happens when the future is created, so the network stack will begin talking to the remote host at that point, and this happens concurrently with whatever code runs the task runs next.

†Note: tokio::net::TcpStream::connect calls mio::net::TcpStream::connect which calls std::net::TcpStream::connect which calls libc::connect, at least on Unix. (Some intermediate stack frames omitted for conciseness.)

1 Like

This might be true for some raw Futures (e.g. connect, as you noted), but it's not true in general, and it's not true for async fns (which are always delayed).

So as soon as there's even the slightest bit of abstraction (e.g. an async fn calling connect), things will be delayed.

That's why I asked if you had any ideas for preemption, since it will be necessary to manually preempt Futures in Rust.

And that manual preemption is necessary with both implicit await and explicit await. In other words, explicit await doesn't really help much.

1 Like

Ah, yes. That's very unfortunate. Sorry, I don't have any suggestions for fixes, but I agree this needs a solution, in particular if Rust is to be used in an environment like the one I was describing (where handling a single request might require async calls to dozens of micro-services, and latency is a high priority).

2 Likes

My point is, in the current design, with already a lot of tooling and crates around, they are a regular type. To make them otherwise, a step back to a design table would have to be made, a lot of existing stuff thrown away.

To be clear, that already needs to be done: most things are based on Futures 0.1, but they will need to move to Futures 0.3 (which is a breaking change).

In addition, we're not talking about changing anything about Future, just the behavior of async (which is not stabilized, and isn't as widely used as Future is). So existing stuff does not need to be thrown away.

1 Like

Something like this should work (I did some light testing):

use std::pin::Pin;
use std::task::{Poll, LocalWaker};
use futures_util::future::FutureExt;

pub struct Force<A> {
    future: Option<Pin<Box<A>>>,
}

impl<A> Force<A> {
    #[inline]
    pub fn new(future: A) -> Self {
        Self {
            future: Some(Box::pin(future)),
        }
    }
}

impl<A> Unpin for Force<A> {}

impl<A> Future for Force<A> where A: Future {
    type Output = Deferred<A>;

    fn poll(mut self: Pin<&mut Self>, lw: &LocalWaker) -> Poll<Self::Output> {
        let mut future = self.future.take().unwrap();

        match future.poll_unpin(lw) {
            Poll::Ready(value) => Poll::Ready(Deferred {
                state: DeferredState::Ready(Some(value)),
            }),

            Poll::Pending => Poll::Ready(Deferred {
                state: DeferredState::Pending(future),
            }),
        }
    }
}


enum DeferredState<A> where A: Future {
    Ready(Option<A::Output>),
    Pending(Pin<Box<A>>),
}

pub struct Deferred<A> where A: Future {
    state: DeferredState<A>,
}

impl<A> Unpin for Deferred<A> where A: Future {}

impl<A> Future for Deferred<A> where A: Future {
    type Output = A::Output;

    fn poll(mut self: Pin<&mut Self>, lw: &LocalWaker) -> Poll<Self::Output> {
        match &mut self.state {
            DeferredState::Ready(value) => Poll::Ready(value.take().unwrap()),
            DeferredState::Pending(future) => future.poll_unpin(lw),
        }
    }
}

You use it like this:

async fn foo() {
    let x = await!(Force::new(bar()));

    ...

    let y = await!(x);
}

Basically, Force::new returns a Future which immediately polls bar(), and then you can later retrieve the value by awaiting on x.

This does have some overhead (a Box heap allocation), but perhaps that can be fixed with some trickery.

I’m not actually sure how to express this with implicit await. Maybe something like this?

async fn foo() {
    let x = async { Force::new(bar()) };

    ...

    let y = await!(x);
}

I haven’t looked too deeply at the various implicit async proposals, so I’m not sure how they handle deferred Futures.

I know they usually use async { ... } to do the actual deferral, but then how do they await the deferred Future?

That’s the reason I used async || { .. }. The idea in “implicit await” is that the explicit async creates a deferred execution, which is equivalent to a closure in sync code.

Personally, in a implicit await I think I’d write that as

async fn foo() {
    let x = Async::start(bar);
    ..
    let y = x.await();
}

I do think the system requires a type (trait) for “synchronous asynchronous execution” which is async fn and a separate type (trait) for “deferred asynchronous execution” which is async || { .. }.

1 Like

I like this. It feels very clean, and it’s shorter than the explicit await.

I also like the strict separation between auto/non-auto Futures, with Async::start and .await() to convert between them.

As I understand it, Async::start and .await() are not magic at all, they are ordinary structs/methods.

Defining the .await() method seems fairly trivial, but how would the Async::start function be defined? It would have to take in the return value of an async fn, but without the auto-await.

I guess it could be used like this instead:

let x = Force::new(async || bar());

That seems quite reasonable as well.

Actually, I don’t think it’s necessary to use the async || { ... } syntax.

The reason is because await does not work inside of closures (unless the closure is async, of course). In other words, async/await is shallow, just like JavaScript async/await.

So in order to prevent the auto-await, an ordinary function works just fine:

async fn foo() {
    // bar() is not awaited
    let x = || bar();
    ...
    // x() is awaited
    let y = x();
}

So that means we don’t need two traits! We can use impl Future for both the auto and non-auto Futures! And async fns can return impl Future!

Any delaying of Futures is simply done with closures, which is already the standard way to delay synchronous code!

I was pretty ambivalent about implicit await before, but this is really making me like it.

And of course the Deferred struct can be changed to have a completely non-magical await method which simply returns impl Future:

enum DeferredState<A> where A: Future {
    Ready(Option<A::Output>),
    Pending(Pin<Box<A>>),
}

impl<A> Unpin for DeferredState<A> where A: Future {}

impl<A> Future for DeferredState<A> where A: Future {
    type Output = A::Output;

    fn poll(mut self: Pin<&mut Self>, lw: &LocalWaker) -> Poll<Self::Output> {
        match *self {
            DeferredState::Ready(value) => Poll::Ready(value.take().unwrap()),
            DeferredState::Pending(future) => future.poll_unpin(lw),
        }
    }
}

pub struct Deferred<A> where A: Future {
    state: DeferredState<A>,
}

impl<A> Deferred<A> where A: Future {
    #[inline]
    pub fn await(self) -> impl Future<Output = A::Output> {
        self.state
    }
}

And now it can be used like this:

async fn foo() {
    let x = Force::new(|| bar());
    ...
    let y = x.await();
}

This is really great! Quite a bit nicer than explicit await.

2 Likes

I haven't read this whole thread, but I wanted to point out some interesting research on this topic. Jaeheon Yi and Tim Disney, working with Freund and Flanagan, studied an extension to Java with "cooperative concurrency" -- where you explicitly identify synchronization points with yield. The idea of their system was that the compiler put in enough locks that this was the only point you could observe "nonatomic execution".

If memory serves, one of the things they looked at was how quickly people fixed threading bugs. They found that, indeed, people fix bugs faster if they have explicit yield.

Just a datapoint.

(I don't think it's the only consideration, mind you, but it is an argument in favor of explicit await.)

13 Likes

Hi, I just find this discussion about implicit await. I’m reposting my comment on Github. Just want to know if it’s a good idea to be explicit about implicit await since there is no feedback so far there.

We can first have prefix await landing:

await { future }?

And later we add something similar to

let result = implicit await { client.get("https://my_api").send()?.json()?; }

When choosing implicit mode, everything between {} are automatically awaited.

This has unified await syntax and would balance the need for prefix await, chaining, and being as explicit as possible.