Pre-RFC: Await generators directly


#1

Summary

Make the await operation available directly to generators, independent of the way they are used- whether as Futures, Iterators, or application-specific types. Provide generator items instead of generator expressions. Limit the use of traits like Future to places they are actually used in the program source.

(The networking WG is currently putting together an RFC for async/await syntax. This is intended as early input to that process.)

Motivation

The current implementation of generators provides closure-like generator expressions as the sole method of constructing a generator value. The futures-await crate wraps this in #[async] functions which return impl Future, and provides the await! macro as the sole method of awaiting a generator value.

However, the await operation is also useful in many other contexts: iterators, GUI event handlers, game logic, interrupt handlers, etc. Many of these don’t need the Future trait or the Context/Waker system it implies.

We can make generators more useful in non-futures-based contexts by providing these features directly to generators- generator items instead of functions returning impl Future, and an await operation that applies directly to those generator items.

Guide-level explanation

A generator or async function is a function that can be suspended and resumed part-way through its execution. For example:

async fn counter(limit: u32) yield u32 -> u32 {
    let mut i = 0;
    while i < limit {
        yield i;
        i += 1;
    }
    limit
}

A generator cannot be called from a non-generator function. It can be called from another generator, or be used to construct a Generator value. Calling a generator causes everything it yields to appear as if it were yielded from the caller:

async fn two_counters() yield u32 {
    let x = counter(1);
    println!("{}", x);
    let y = counter(2);
    println!("{}", y);
}
let c = two_counters::new();
assert_eq!(c.resume(), GeneratorState::Yielded(0));
assert_eq!(c.resume(), GeneratorState::Yielded(0)); // also prints 1
assert_eq!(c.resume(), GeneratorState::Yielded(1));
assert_eq!(c.resume(), GeneratorState::Complete(())); // also prints 2

Traits like Future, Stream, and Iterator are implemented for wrapper types, to provide APIs specific to those contexts:

// generator yielding Async<T> implement IntoFuture:
tokio::run(async {
    let page = get_page_text("https://example.com/");
    // ...
});
// generators yielding T and returning () implement IntoIterator:
for i in two_counters() {
    println!("{}", i);
}

Reference-level explanation

Generator types and values

A generator item is syntactically a function prefixed with the async keyword. It can also specify a type to yield, which like the return type defaults to () if unspecified:

async fn generator(t: T, u: U) yield V -> W { ... }

A generator expression is syntactically a closure with no arguments, prefixed with the async keyword. It can also specify a type to yield the same way, and evaluates to a value with an anonymous type:

async yield V -> W { ... }

A generator item declares a type. That type, and the anonymous types of generator expressions, implement the Generator trait, which now also provides a single static method for construction. For example, the counter generator above is equivalent to this:

struct counter(...);

impl Generator for counter {
    type Args = (u32,);
    type Yield = u32;
    type Return = u32;

    extern "rust-call" fn new(args: Args) -> Self { ... }
    fn resume(&mut self) -> GeneratorState<Yield, Return> { ... }
}

A generator body has two additional capabilities beyond a normal function body:

  • It can yield a value to its resumer, as in today’s implementation.
  • It can directly invoke another generator as if it were a function, which awaits it. This constructs a generator value of that type and resumes it repeatedly, yielding what it yields, until it completes, when it evaluates to its return value.

Generator methods

Inherent impls and trait impls can contain generator items. They behave much like associated types, though awaiting them looks like a method call:

trait Trait {
    async fn g(&self) yield i32;
}

async fn async_caller<T: Trait>(t: T) yield i32 {
    t.g();
}

fn caller<T: Trait>(t: T) {
    let g = T::g::new(t);
    let _: GeneratorState<i32, ()> = g.resume();
}

This does mean that generic generator trait methods are blocked on generic associated types.

Drawbacks

  • Somewhat more work to implement than converting futures-await's macros to language features.
  • More design work to be done around async for- see below.
  • Probably more.

Rationale and Alternatives

Generator composition

The main alternative design today is represented by the futures-await crate. There, only futures can be awaited, and any other use of generators must resume its callees in a loop manually.

This design lifts that restriction, and also makes it easy to bring futures back into a generator context- Future might have a method async fn await(&mut self) yield Pending -> Output that contains such a polling loop.

Debugging and optimization

This design can improve the debugging experience. Awaiting a generator no longer goes through a wrapper type’s poll function, allowing debuggers to step directly into the callee. Further, stepping into a resume() can know to break only in the inner-most generator.

This design is also partially forwards-compatible with optimizations like a CPS transform for generators. (The dependence does not go the other way- we do not need anything from the CPS transform design to get the benefits of this design.)

API design

This design also neatly sidesteps two sources of potential confusion:

With futures-await, you can call an async function from a synchronous function, and get back a future. You might reasonably look at Future's methods to try to extract the value, or even get an “unused value” warning. But with this design, you must either invoke the async function from an async context, or intentionally construct a generator value from it.

With any futures-based await syntax, there is a question of whether async functions should include impl Future somewhere in their return type or declaration. C# async functions and C++ coroutines do include this information; Kotlin coroutines do not; Python and Javascript don’t specify types at all. This design decouples generators from futures so this question does not arise at all.

It may not even be necessary to put Future in the standard library to get async syntax (though it may still be desirable for other reasons).

Unresolved questions

yield

As described above, a generator is suspended by yielding a value. Another option, potentially more flexible, might be something that looks like call-with-current-continuation:

suspend me {
    /* me is a reference to the outer-most generator */
    /* this block's value is yielded */
}

Another question around yield is whether it should return a value on resume, and if so how. The first resumption of a generator does not have matching a yield expression. This might be solved by running the first segment of a generator into its new method, but this complicates self-referential generators and reduces the benefit of this design’s clear separation between construction and resumption.

async for

Two traits from the futures crate neatly represent the async world’s versions of values and iterators- Future is to T as Stream is to Iterator<Item = T> (assuming Future's Item/Error are combined into a single Output). However, generators introduce another trait. Generator is somewhere in between and below these two traits, depending on how you look at it.

I see a couple of ways to integrate generators with Future: a blanket impl for generators that yield (), or to better express intent, a blanket impl for generators that yield some marker type- perhaps a ZST struct Pending;.

This design could be expanded to Stream as a blanket impl for generators that yield Option<T>, or again an equivalent marker type. But this starts to feel kind of messy- how do you await a normal Future-style generator here, since those yield ()? Some kind of ?-operator-like conversion? How does that interact with the potential for a CPS transform?

And worse, how do you do iterator combinators here? What’s the equivalent for IntoIterator when you write async for? It feels like what we really want is a new trait, equivalent to Iterator but with an async next method:

trait AsyncIterator {
    type Item;
    async fn next(&mut self) yield ??? -> Item;
}

But then we’re back to hand-writing the iterator-level state machine. Is the answer yet another operation, like futures-await has with #[async_stream] and stream_yield!?

Perhaps, by analogy to a hypothetical const impl, we could even reuse the same trait Iterator in both contexts.

Syntax

I kind of like the lack of an extra await keyword, but really all the syntax here is up for bikeshedding as far as I’m concerned. It’s not really the point of this pre-RFC.


Explicit future construction, implicit await
#2

Why make this “await” functionality built-in when it can just be written explicitly as a loop that calls the “awaited” generator and yields?

Non-future based contexts can already use the generator functionality directly, and maybe a propagate_yield!() macro could be added if this yield propagation pattern turns out to be frequently used.

More fundamentally the fact that something that looks like a function call actually automatically yields seems totally unacceptable (Rust seeks to make all such control flow explicit, see for instance the “?” operator), and also it would only work if the yield types match (with no way to provide a transformation function, for instance).

I general, not clear why this would be better rather than worse compared to the current generator syntax with async/await built on top as a library/plugin.


#3

The usual argument for making await explicit, in any language, is that it makes it clear which calls can suspend the current coroutine (potentially blocking, and potentially allowing other coroutines to alter state) and which can’t. As @jmst mentioned, that seems to fit pretty well with Rust’s existing decision to make unwinding points explicit.

However, I can see the argument for making await a builtin feature that can be used in more scenarios. Dedicated syntax could also be more succinct: await foo() vs. await!(foo()).


#4

That’s covered in the rationale section- making it built-in makes it easier to compose generators, and makes it easier to write good tools for, and makes it easier to switch around the calling convention in the future. There’s a single, canonical “await” operation that can be e.g. put into debug info or compiled down to something other than a loop.

For that matter, I believe it was already intended for await to become built-in, regardless of what I wrote here. All I’m proposing is to make it built-in for generators rather than for futures.

As far as syntax goes, making it explicit like await foo() doesn’t alter the main point I’m making here. I certainly understand the analogy to ? and the benefits of explicit suspension points.

Three reasons to make it implicit:

  • The explicit await some_future.await() is redundant.
  • The function is already marked async. unsafe functions and blocks don’t mark every unsafe operation they contain, it’s enough to mark the whole thing once.
  • An awaited future is always (often?) a Result, which means it’s often used with ?. The precedence of await foo()? is confusing; (await foo())? is ugly; await(foo())? is hardly better than await!(foo())?.

Perhaps if it is explicit it should be another suffix operator, to preserve the order of the chain of operations? It does look rather strange that way (foo() await ?, or foo()@? if you can stomach the sigil (I can’t)).

Yes, it only works if the yield types match. But this is a fundamental aspect of await, regardless of how it’s implemented.

There is a way to provide a transformation function- build the generator explicitly and await an async map method on it. This is the exact same solution as with futures, just without tying yourself to that particular trait.


#5

async/await nomenclature seems very tied to the futures use-case of generators. The auto-yielding of sub-generators also seems only useful for generators implementing Future specifically (I believe neither the Iterator nor the Stream use cases would want this), for example this seems odd to me:

async fn range(start: u32, end: u32) yield u32 -> () {
    for val in start..end {
        yield val;
     }
}

async fn range_twice(start: u32, end: u32) yield u32 -> () {
    range(start, end);
    range(start, end);
}

What I have been using for this so far is a yield_from! macro (inspired by Python’s yield from syntax). I wonder if there’s a way to have a general yield from or similar syntax for generators that can be built on to offer a more specific await for futures.

There’s also the difficulty with futures 0.2 that somehow the context argument needs to be passed down. I’m still hopeful that we will get “generator arguments” back at some point to allow this, have you considered how this might be compatible with a trait similar to

trait Generator {
    type Args;
    type Yield;
    type Return;

    fn resume(&mut self, args: Args) -> GeneratorState<Yield, Return>
}

#6

Notably, Python’s yield from and await are almost exactly the same internally- await is a later addition that performs the identical operation after a type check. So however it looks syntactically it’s certainly valuable for both futures and iterators, with the same arguments as above for leaving it implicit.

That could certainly work. We would need to figure out the disparity between the first resumption and later resumptions. Perhaps by running the first segment as part of construction and passing the context as an argument as well? Or by repurposing the argument list to describe the values passed into resume? Or by giving a name to the yielded value as part of the declaration:

async fn f() yield(cx: &mut task::Context) Pending -> Output {
    /* cx is a new binding each resumption */
}

Or perhaps even using the call/cc-like syntax, if we don’t want to follow the model of yield returning a value:

async fn f() yield(&mut task::Context) Pending -> Output {
    suspend me, cx {
        /* me is the generator */
        /* cx is the value passed into resume */
    }
}

Of course there’s always thread-locals, which is how futures-await currently works even under futures 0.2.


#7

After reading the relevant PEPs (PEP 255 for the basic generators, PEP 342 for the addition of sending arguments in, PEP 380 for the addition of yield from) I think there’s just one extension to your proposal that would be needed to nicely support resume arguments:

To be able to await/yield from/implicitly wait for a sub-generator, it must have compatible yield and resume types.

Specifically given the trait definition at the end of this post it must be some Generator<Yield = Self::Yield, Resume = Self::Resume>.

This comes almost directly from how yield from works in PEP 380 (minus some minor back-compat concerns), all arguments passed in are delegated directly to the sub-generator and yielded values are yielded back. The Yield types matching also appear to be an un-mentioned requirement of your original post.

This results in a trait

trait Generator {
    type Args;
    type Resume;
    type Yield;
    type Return;

    extern "rust-call" fn new(args: Self::Args) -> Self;
    fn resume(&mut self, args: Self::Resume) -> GeneratorState<Self::Yield, Self::Return>;
}

You can see how this can be used to define Iterator, Future or Stream implementations in this playground.

As you mentioned, there is still a relatively major unresolved question with argument taking on how to get the first argument in to the generator. Python was forced to basically have the equivalent of fn resume(&mut self, args: Option<Self::Resume>) and say that the first call must pass None and all remaning calls should pass the actual arguments. I hope it’s possible to come up with a better plan for this without backwards compatibility concerns. (I should look back at the old implementation and discussion on the original generators RFC at some point…)


Anyway, the whole point of this post is that your current proposal is definitely forwards-compatible with supporting generator arguments in the future (well, as long as the trait remains unstable anyway).

It should also be forwards-compatible with CPS (although I must admit I haven’t really followed the details of it in your post fully). Since all the resumed and yielded types are compatible at all levels in a single CPS transformed generator stack the only incompatibility between the function types is the Self type, same as without any arguments, so however that’s being handled should be capable of handling the arguments easily.

thread-locals...

Of course there’s always thread-locals, which is how futures-await currently works even under futures 0.2.

Yeah, great for those of you with runtime threading libraries :cry:, I’m currently going through the pain of trying to work out a not-horrible method of making futures-await safe to use on no_std that will actually have a chance of being merged back upstream. Super easy to just ignore thread-safety when I can guarantee that I won’t use threads/interrupts, but just promising isn’t good enough for core libraries.


#8

Why not just pass Resume as an argument to the generator?

That is, code would be like this

fn make_generator(a: A, b: B, ...) -> impl Generator<Resume = Res, Yield = ..., Return = ...> {
    move |r: Res| {
         ...
         let r: Res = yield ...;
         ...
         let r: Res = yield ...;
         ...
    }
}

Putting Args and new in the Generator trait itself seems wrong: a GeneratorFactory trait or a simple closure can be used if someone wants to abstract over them (just like Future and Iterator don’t have “new” methods).

It may seem to be possible to emulate Resume by yielding a closure that takes the Resume value, and then assigning that to a variable, but I think that to pass the borrow checker it would at least need an extra <'a> parameter on Yield (so the closure can die after resuming so we unborrow the variable), which requires ATC; this also requires the caller to store the closure, which requires it to be self-referencing, essentially requiring it to be another generator; so it seems we need Resume and can’t emulate it.


#9

That’s one possibility, but then you’ve lost the ability to give generators a different set of initial arguments, outside of just using a function returning impl Trait, which makes the actual type unnameable. (Unless you also add typeof or abstract type.)

Splitting new into a separate trait is fine- it just needs to be in some trait so you can name the type you’re working with.


#10

I’m working on an RFC for async/await syntax (as you may already know), and I definitely considered something in this space, where an async function and a generator are just the same thing (you may know this too, since I posted a gist on twitter a few months ago).

Ultimately I’ve moved away from that kind of unified design for a few reasons. First, I think users find it easier to model async and generators as separate functionalities when learning than to understand them as isomorphic. In contrast, I think the main benefit of unifying them is aesthetic, which isn’t supremely compelling.

The biggest reason not to do this, I think, is that I think having two separate functionalities makes it very easy to model streams, which currently are a bit of a problem syntactically (@Nemo157’s alternative pre-RFC has a proposal, but I’ll respond there why I’m unconvinced about that).

If we have both async fn() and generator (let’s call that fn() yield, we have a matrix:

Evaluated immediately Evaluated asynchronously
Return once fn() -> T async fn() -> T (Future)
Yield many fn() yield T (Iterator) async fn() yield T (Stream)

#11

Completely agreed. I kind of danced around this in my unresolved questions but never really figured it out. That’s a good way to put it.

I’m not really interested in mushing the two operations together, really- just making them available without depending on Future+Context+Waker. Perhaps that’s not completely realistic, and maybe it’s better to push directly on things like easier no_std futures and better debugging, but I’d like to keep trying a bit longer.

We could borrow from the concepts of effects or scoped continuations (conceptually, not talking about CPS here): treat “suspending because we’re not done yet” and “suspending because we’ve generated a value” as two independent+orthogonal operations; give them two different names and two different ways to interface with from the outside. (This sounds like where your RFC is going anyway?)

I also like the C# team’s point (from @Nemo157’s RFC thread) that the explicit inclusion of Task in the return type is for polymorphism of the await mechanism- I had noticed that as well. While we won’t need to separate StableFuture+Future+Box<Future>, I do still think awaiting could be useful outside of the actual Future trait.

As a summary: I think what’s left over from this proposal, given that we don’t want “generators are async functions are streams,” is this:

  • The ability to swap out the await mechanism, not by changing a return type like C# or C++, but by using different wrappers/drivers.
    • In particular, eliminating extra poll-like stack frames between nested generators/async fns/etc.
    • This leaves the job of control flow to the compiler (making it easier to optimize and debug through) and the job of (interfacing with the) scheduler to the wrapper.
  • Some kind of yield from-like capability that isn’t just defined to desugar to a loop { .. yield .. }
    • In other words, a way to compose fn() yields, and I suppose also a way to do both at once and compose async fn() yields.
    • This, again, leaves the job of control flow to the compiler and the job of scheduler to the wrapper.
  • The idea that these functions are only callable from the appropriate context, and that they must be instantiated some other way to cross the boundary.
    • I expect it would help users’ mental models and learnability not to have the syntactic option of “calling” an async fn outside of an async block.
  • Bikeshed-y syntax: when you’re in a particular context, calling another function from that context shouldn’t require any extra await syntax.
    • Given the idea of wrappers/drivers above, await syntax would lead to await a_future.await() (unless futures turn around and implement the native trait for the async fn context).
    • The function itself already specifies what context it’s part of. unsafe doesn’t mark every individual operation either, that level of granularity should be enough.
    • Any await syntax will frequently be used alongside ?. await foo()? has wrong or unintuitive precedence; (await foo())? is ugly; await(foo())? has the same problems as try!() that drove us to ?; foo() await ? is strange; foo()@? is too many sigils.

Finally, how feasible would it be to reuse the solutions for Iterator and Future by combining them, rather than just using a new trait Stream? Some kind of async impl Iterator maybe? Or maybe it’s not worth it.