Do we need `CoroutineState`?

libs
1

trait Coroutine, enum CoroutineState

Originally (futures@0.1), the Future trait baked in fallibility, with Future::poll returning Result<Poll<Self::Item>, Self::Error>. As part of uplifting into std, this was removed, and Future::poll now returns Poll<Self::Output> instead, with fallible futures using Output = Result<Item, Error>.

Are coroutines similar (i.e. shouldn't bake CoroutineState into the trait signature) or different (e.g. since they don't have Poll to indicate completion like futures do) here?

A key observation is that from the outside, Coroutine::resume looks the like any other subroutine; the underlying semicoroutine[1] has been transformed by the compiler to fit the usual stack structured control flow. It's for this reason I think the desirable primitive shape could look more like a FnPin trait[2], but I'm not particularly interested in discussing any naming/syntax except as it relates to the signature of Coroutine::resume (FnPin::call_pin).

The primary argument for -> CoroutineState<Self::Yield, Self::Return>, as I understand it, is that Coroutine encodes that it is a logic error to resume after it produces CoroutineState::Returned(_) (like with Iterator or Future producing None or Poll::Ready(_), respectively). If resume just returns Self::Output, there's no way for generic consumers of Coroutine can't know when a coroutine is exhausted.

But... is this actually an issue? FnMut has this same "issue" but it doesn't seem to be much of an issue in practice; APIs which take impl FnMut specify how they expect the callback to behave and everything seems to work out fine there.

There's a straightforward option for many yield closures: a call after a return resumes from the top of the closure. I really like this model conceptually because it means using yield doesn't change what return does. However, yield closures which move from captures (i.e. would be FnOnce if not yielding) cannot do so and need to poison any future resumption to panic after a return, and resuming after an unwind is yet another can of worms / potential divergence from non-yielding closure semantics (and potentially involving the unclear domain of UnwindSafe).

Of course, a coroutine with Yield=Output, Return=! is isomorphic to (and implicitly becomes the case for a looped yield closure body), meaning this should be a purely API design question. For non-yielding closures, the difference of "(typically) can call again after return" is directly represented with FnMut versus FnOnce.

The simplest option is, of course, to keep the machinery of the coroutine transform internal to the compiler and never expose it. Generators are a more straightforward API surface (well...[3]) to expose than coroutines, and every other language I can think of is generally happy using generators and/or async, using shared mutability (e.g. channels) to inject any "resume arguments". As such, maybe Rust doesn't need to invent a stable-ready solution for borrow verified semicoroutines.

  1. Terms:

    subroutine
    single-entry, single-exit
    semicoroutine
    multiple-entry, single-exit
    coroutine
    multiple-entry, multiple-exit
    Note: entry/exit is referring to the calling convention. Single-entry means you only enter a routine at a single point/label; single-exit means you only exit back to the caller by paired call/ret. (TCO breaks subroutine convention but imperceptibly to the caller.) A structured call stack is so ingrained to modern thinking that people usually mean semicoroutine when they say coroutine. ↩︎

  2. e.g.

    trait FnPin<Args: Tuple>: FnOnce<Args> {
    extern "rust-call" fn call_pin(self: Pin<&mut Self>, args: Args) -> Self::Output;
}

// is Unpin bound needed? cf. future::poll_fn
impl<A: Tuple, F: FnMut<A>> FnPin<A> for F;
impl<A: Tuple, F: FnPin<A>> FnMut for Pin<&mut F>;
    ↩︎

  3. In that they they unambiguously should yield the iterator output, return () for exhaustion, and panic if resumed after return, at least. Needing to handle trait coloring of un/pinned &mut self is a problem for generators but not for coroutines (Pin<&mut impl ?Unpin> accepts both cases). Lending (GAT fn(&mut self) -> Self::Output<'_>) is a question for both, but again worse for generators than coroutines, since the coroutine trait could bake in that its output potentially borrows from self and there aren't any additional issues with representing input-derived borrows beyond the existing difficulties with higher-ranked Fn closures. ↩︎

2

This is what I hope we do, for the foreseeable future anyway. I think we end up with a simpler model if we don't expose a version of coroutines which has both yield and return. I'd love to encode in the type system that they don't have any concept of a return value/type at all.

That said, if we do end up having coroutines, I can imagine ways to encode the type such that it's impossible to call them again once they've returned, but the complexity of doing that may outweigh the benefits. Going with convention, similar to Iterator (don't call once it returns None), will probably suffice.

3

It would be nice to have an operation that borrows the “essence” of a value: like a &mut but dropping it drops the value in place and the only way to regain access to the value is through a give-it-back function. This way, it would be quite straightforward to make a resume signature that ensures proper cleanup of the coroutine and makes it impossible to call again after it has returned.

4

If you have an Option<T> you can do that with Option::take.

There's no safe way to do that for a type that doesn't have the equivalent of None.

5

I agree that this is currently not possible. But it would be possible if Rust had a third kind of reference: “owned reference”. Box isn’t the same thing because owned references need not be restricted to heap allocations.

Also, the borrowed value would need to stay unavailable (e.g. borrowed) until reconciled with the reference to pass back ownership.

2 Likes
6

I am very much interested in having a concept of "owned reference"; this has come up many times and been discussed for years. The distinction I was making was that the original value that gets turned into the owned reference would need to have one of two things: either a runtime-visible way to take the value such as a type like Option that can be taken, or ownership that "passes into" the reference such that the original value becomes statically inaccessible the same way it does when you move out of it.

2 Likes
7

Fwiw, that's a bit more permissive than "just don't expose coroutines" actually, since it still allows writing coroutines (and not exclusively generators). I'd be on board — the "resume from top" model essentially works by saying return $expr is yield $expr; continue 'loop (modulo the details of how resume arguments are accessed[1]) instead, and actually writing that for clarity isn't a bad idea. The one wrinkle is of course the "tail expression return;" perhaps requiring ! makes it easier to justify unwinding poisoning the coroutine to panic on any further resumes.

This version actually reasonably excites me, as it seems more likely to be "stable worthy" than any of the others, imo. Essentially, requiring Coroutine::Return = ! mitigates a lot of unclear questions about semantics, the remaining questions being handling resume arguments and distinguishing coroutines from generators.

  1. I continue to be a fan of the "magic mutation" model, where this equivalence holds more directly; it's weirder with the "yield produces" model which treats the initial resume arguments differently. ↩︎

2 Likes
8

I'm reminded of this old post of mine:

(and the thread) as well.

1 Like