Syntax for generators with resume arguments

Not a very baked idea at this point, but what if syntax for generators with resume arguments were something like this:

generator {
    resume x,y,z;
    ...
    yield x resume u,v,w;
    ...
    while u < v {
        ...
        yield a + b * c resume u,v,w;
        ...
    }
    ...
    return w;
}

  • All resume's must bind the same number of variables.
  • Variables bound by resume live until the next yield.
  • resume must be the first statement of the generator body, and must follow each yield (but not return). It may be omitted for argument-less generators.

I think this nicely side-steps the issue of unpacking tuples returned by yield and is also less surprising than implicit re-binding after each resumption.

Previous discussions involving this topic:
[RFC] Unified coroutines a.k.a. Generator resume arguments
[Post-RFC] Stackless Coroutines
Pre-RFC: Generator resume args

If we want this style of syntax, here's an alternative that avoids the resume semikeyword on yield statements: yield $expr => a,r,g,s;.

Note that this syntax would require an edition boundary to turn at least generator into a keyword. resume could probably get off as a weak (context-sensitive) keyword, but I'm not exactly certain.

The main problem with resume arguments is the asymmetry between the initial control flow entrance and resuming at yield points. I don't think any syntax can possibly be "obviously correct" unless it corrects for that incongruity.

Personal opinion

I still personally think that resumable closures ("argument re-binding") is the best approach, and am slowly collaborating on an RFC putting forward that position with its full picture. Not to derail this thread, though.

The TL;DR is that || return 5 and || loop { yield 5 } are theoretically equivalent, so what falls out of actually unifying them, and |x| return x with |x| loop { yield x } as well?

1 Like

Yes, but I think some new keyword for generators is pretty much unavoidable, unless we want to go with something like yield fn { ... }.

And the above does that, does it not?

But a lot of people felt that was too magical, hence this proposal.

There's still asymmetry between the initial resume $arglist; and later yield $expr resume $arglist.

Unless you'd like to suggest separate yield and resume statements?

And the version of that proposal I'm working on is quite unmagical, so long as I can explain it correctly. Every proposal in this space is matter of what direction you look at it from.

If some kind of explicit resume syntax will be made, I have a general opinion on its syntax (I still prefer the implicit way).

Because the "resume argument" position expects a pattern (it can be restricted to the simplest variable patterns but semantically it is a pattern, not an expression), it should be similar to the existing syntax expecting a pattern.

yield $expr => arg is not good at this because existing match arm syntax $pat => $expr expects the pattern at the left-hand side of "=>", not the right-hand side.

I don't have good alternative syntax, though. Maybe let $pat yield $expr or something.

Not quite. I believe it would take FnArgs, not Pat. (i.e. a,r,g,s is not a pattern, (a,r,g,s) is; a,r,g,s is an argument list (without types).

I meant that the argument position syntax is more of a pattern than an expression, relatively. I know that is not precisely a pattern, but with the type ascription pattern, an argument list (in a closure, ignoring self thing) is a list of patterns separated by commas.

What's wrong with let (u, v, w) = yield x; ?

2 Likes

How would you get the resume arguments before the first yield?

You could just treat it as a normal function call until the first yield.

However, I do like @CAD97 proposal of modifying the parameters.

I wrote a blog post on the first-yield/last-yield asymmetry a while ago if people are curious about some of the difficulties. @vadimcn's proposal handles it pretty well on the whole! It suppors multiple arguments, explicitly shows the location of the first resume, and it's even compatible with "magic mutation" in the sense that the resume ... postfix could control which parameters are re-assigned across yields. Magic mutation used to scare me but I now think that it is the right approach for a few reasons:

  • It isn't really mutation. Think of it more like re-assignment of the parameter binding.
  • It does allow parameters to be moved. Just as let x = ...; loop { drop(x); x = ...; } is allowed, so is |x| { drop(x); yield; }.
  • In practice, generators rarely care about old inputs. Futures are strictly forbidden from using old contexts, streams tend to fully process each item after moving onto the next, etc. Usually people yield because they are finished with whatever they already have and need something new.
  • I can do stuff like |c: char| while c.is_whitespace() { yield; } (or generator { resume c; while c.is_whitespace { yield resume c; } } under this proposal) which becomes really second-nature after a while.
  • Rust is really good at preventing odd mutations at odd times from having odd effects. If you try to hold a reference to the input across the yield, you will get told to move it to a dedicated binding first.

The proposal myself, @CAD97, and @pcpthm are working on is here if people have feedback. The more input the better!

Meh... what's wrong with

|mut c: char| while c.is_whitespace() { c = yield; }

That makes it much clearer that c is changing.

Edit: But I do look forward to seeing the details of the proposal you come up with.

3 Likes