Syntax for unifying generators, async/await, etc

This is not even a pre-RFC, I just wanted to present a few ideas for how these concepts could be unified in a clean way without significantly complicating the language, mainly focusing on syntax, as that's how users will interact with it.

Unstable coroutine syntax

This is a generalisation of the state-matchine-transform performed by the Rust compiler. It's not necessarily expected that this syntax would ever become stable: if it does it would be some time after the surface syntaxes have stabilised.

Everything that requires a state-machine-transform should de-sugar to a coro block. Anything that does not require such a transform should not. The coro statement is the state-machine transform if you will.

coro a: i32, b: i32 {
    yield a + 1;
    yield b + 1;
    panic!() // the last expression in a `coro` block must have type `!`
}

a and b are "reassigned" on each yield based on the resume arguments. I consider this magic acceptable because the coro keyword is clearly visible: this magic will never happen without the coro keyword explicitly written, even in the various surface syntaxes I will describe later.

The alternative of having yield evaluate to the resume arguments has many downsides, the most obvious of which is that the initial resume arguments must be acquired some other way, which would be inconsistent. However, there are several other issues too.

I would expect IDEs to highlight uses of these names in a similar way to mutable variables, so that this magic is very obvious.

Async/await (Future) desugaring

This desugaring happens when an async block is used.

async {
    let x = foo().await;
    x + 1
}

=>

CoroFuture(coro ctx: &mut Context {
    // Not showing the desugaring necessary to pin the temporary which
    // stores the return value from `foo()`, as it's not relevant.
    let tmp = foo();
    let x = loop {
        match tmp.poll(ctx) {
            Poll::Pending => yield Poll::Pending,
            Poll::Ready(res) => break res,
        }
    };
    yield Poll::Ready(x + 1);
    panic!("async fn resumed after completion")
})

Generator (Iterator) desugaring

This desugaring happens automatically in functions and closures containing a yield statement.

|x| {
    yield x + 1;
    yield x + 2;
}

=>

|x| CoroIterator(coro {
    yield Some(x + 1);
    yield Some(x + 2);
    loop { yield None; }
})

Note that iterators created from coroutines can implement FusedIterator.

Async generator (Stream) desugaring

This desugaring happens automatically in async functions and async closures containing a yield statement.

async |x| {
    yield x + 1;
    let y = foo().await;
    yield y + 1;
}

=>

|x| CoroStream(coro ctx: &mut Context {
    yield Poll::Ready(Some(x + 1));

    // Not showing the desugaring necessary to pin the temporary which
    // stores the return value from `foo()`, as it's not relevant.
    let tmp = foo();
    let y = loop {
        match foo().poll(ctx) {
            Poll::Pending => yield Poll::Pending,
            Poll::Ready(res) => break res,
        }
    };
    yield Poll::Ready(Some(y + 1));
    loop { yield Poll::Ready(None); }
})

Interaction with traits and type system

Each coro block defines a unique anonymous coroutine type.

Every coroutine type implements the following unstable Coroutine trait:

trait Coroutine<Inputs> {
    type Output;
    fn resume(self: Pin<&mut Self>, inputs: Inputs) -> Self::Output;
}

Several "special" traits are also implemented for corresponding newtype wrappers around coroutines:

// CoroIterator([anonymous coroutine type])
impl<C, T> Iterator for CoroIterator(C)
where
    C: Coroutine<(), Output=Option<T>> + Unpin
{
    type Item = T;
    ...
}
// CoroFuture([anonymous coroutine type])
impl<C, T> Future for CoroFuture(C)
where
    C: for<'a> Coroutine<(&'a mut Context,), Output=Poll<T>>
{
    type Output = T;
    ...
}
// CoroStream([anonymous coroutine type])
impl<C, T> Future for CoroStream(C)
where
    C: for<'a> Coroutine<(&'a mut Context,), Output=Poll<Option<T>>>
{
    type Item = T;
    ...
}

These trait implementations can be provided by core.

Summary

These design choices would allow early stabilization of the various surface syntaxes, whilst still allowing the full power of coroutines with arbitrary inputs and outputs to be stabilised later on.

4 Likes

Previously: https://lang-team.rust-lang.org/design_notes/general_coroutines.html

4 Likes

I guess the async/await desugaring is missing a loop?

CoroFuture(coro ctx: &mut Context {
    // Not showing the desugaring necessary to pin the temporary which
    // stores the return value from `foo()`, as it's not relevant.
    let x = loop {
        match foo().poll(ctx) {
            Poll::Pending => yield Poll::Pending,
            Poll::Ready(res) => break res,
        }
    };
    yield Poll::Ready(x + 1);
    panic!("async fn resumed after completion")
})

Ah yes, I've fixed the first post.