Rust generators: exploration of potential syntax

[vojtechkral]

@CAD97

I don't want trait Generator<Args> { type Yield; type Return; } . I want trait FnMut<Args> { type Output; } . (And, well, FnPinMut for semicoroutines[1] which require pinning.)

My core argument here is that semicoroutines[1] and FnMut have the exact same interface from the caller's perspective

Ok but desugaring of async blocks (currently) requires both those features (a resume type and a yield type). As far as I know, the GenFuture (the async block future impl type) takes a impl Generator<ResumeTy, Yield = (), Return = T> where ResumeTy is an implementation-detail wrapper for &mut Context. You could probably replace Yield = (), Return = T with just Poll<T>, after all, GeneratorState<(), T> is equivalent to Poll<T>, that one I agree with. The resume type should stay though IMO, although it probably doesn't need a dedicated syntax right away like you & OP said.

OTOH if there were syntax for this, you could use generators to write low-ish level futures easily (by having the context passed in via yield). OTOH2 this would probably not be widely useful and besides I think you can get context out of async blocks today with a crafty Future...

[ekuber]

The same problem applies for the leading yield syntax: yield fn foo() might make sense, but yield || { yield 32; } requires fancier semantics to be able to yield closures: yield || { yield || { 3 } }, what's the type of that expression? There are at least 2 "valid" interpretations.

There's a case to be made about allowing regular closures to turn into generators merely with the use of a yield in them, which is what they already do in nightly, but I can be convinced either way.

The issue with gen or iter the same that goes for all keywords: until the next edition we'd have to preface it with k# (ugly) or introduce a new feature to opt-into new keywords (a new proc macro intrinsic was proposed for this, which I like).

[CAD97]

@vojtechkral

The resume type should stay though IMO, although it probably doesn't need a dedicated syntax right away like you & OP said.

I'm in agreement here. The syntax for resume arguments would just be |arguments| { yield closure }; you'd get FnMut<ResumeArguments> { type Output; }. There's no removal of features in using FnPinMut from Generator. Generator<ResumeTy, Yield = (), Return = T> becomes FnPinMut<ResumeTy, Output = Poll<T>>.

In the future, I want this. But this thread is focused on just the Args = () case to avoid the extra bikeshed of how resume arguments are handled.

@ekuber

The same problem applies for the leading yield syntax: yield fn foo() might make sense, but yield || { yield 32; } requires fancier semantics to be able to yield closures: yield || { yield || { 3 } } , what's the type of that expression? There are at least 2 "valid" interpretations.

There's a case to be made about allowing regular closures to turn into generators merely with the use of a yield in them, which is what they already do in nightly, but I can be convinced either way.

I'm strongly in favor of just adding yield to a closure body giving a yield closure. That's actually part of my reasoning in making yield closures just Fn[Pin]Mut, rather than a Generator or Iterator.

[tema2]

Why not to have all the traits:

• AsyncGenerator
• Generator
• Stream
• Iterator
  Edit:
• Future is also here

And one core FnPinMut trait, which is actually produced by any syntax.

After than we just implement all desired traits for appropriate anonymous types:

• Iterator<Item=Item> ~= FnPinMut(())->Option<Item> + Unpin this is case for gen fn and gen closures;
• Stream<Item=Item> ~= FnPinMut(&mut Context) -> Poll<Option<Item>> - async gen closures and fn s;
• Generator<R,Yield=Item,Return=Return> ~= FnPinMut(R) ->GeneratorState<Item,Return> - closures with both yield and return;
• AsyncGenerator<R,Yield=Item,Return=Return> ~= FnPinMut(R,&mut Context)->Poll< GeneratorState<Item,Return>> - async generators;
• Future<Output = T> ~= FnPinMut(&mut Context)-> Poll<T>- futures.

And after that we are free to add any sugar.

My bet is to have gen and async gen with no distinct Return type for producing streams and iterators (they have no resume arg) and second closures with both return and yield allowed producing either a generator (both present) or a semicoroutine(only one present).

Edit 2: there have been a lot of disagreement on notable things:

1. how to handle coroutines finish, the two real options have been underlined:
   • Implicit restart: we have this for Iterator and Stream as nobody wants to loop their coroutine code for virtually no reason, and get runtime panics if they didn't.
   • Poison state: this goes to closures for FnPinMut and Generator - as of these are "low level" things, explicit looping should be preserved.
2. "yield as an expr." vs "magic mutation": \
   • As of on "high level" we don't really have explicit resume arguments (fold and co. would disagree) don't want cognitive burden of magic mutation, the "yield as an expression" is the best choice; first resume is simply implicitly assigned to arguments binding, which is alive utill next yield occurs.
   • Magic mutation is likely correct and despite it's introducing troubles for borrow checking, "low level" generators and FnPinMut semicoroutines go this way, just as feature works today.

Originally descibed here.