Writing a couple of blog posts about the syntax for “for await” loops - loops that process streams (instead of iterators) by awaiting each element. This thread is for discussing them.
We’re trying to experiment with different discussion forms that keep the conversation more manageable for everyone, so I’d like to try out an experiment here. For clarifying questions and small points of feedback, please feel free to write a post in this thread. But for larger arguments and full alternative proposals, please do us a favor and write a blog post or gist, then link to it from the thread. Make sure to put work into it so that you are making the best case for this position.
We hope this will help us track different alternatives, present each alternative with the best case with less repetition, and keep the thread from exploding into hundreds of responses. Its an experiment, so we’ll see how it goes.
My knee-jerk reaction is: “Please no! Do we really need even more confusion around async/await?!”
First, I think some real world-ish examples and type annotations for elem and stream would be helpful for those of us who rarely work with async programming. Well, and general motivation section as well.
Second, why can’t we use async for elem in stream { .. } to sidestep the issue? (i.e. I don’t find your explanation sufficient) This way async for can be interpreted as a combined keyword and semantics of async for can be more or less separated from semantics of traditional for loop.
You mention attaching await to elem but not to for. for.await elem in iter is another possible choice that I’d’ve liked to see in that section.
For “just make for work without an extra keyword”: we make await explicit on futures, it should be explicit for streams.
My gut idea is that [keyword] for is the option we want, since the asyncness is a property of the stream, not the element. I’ll look at writing up my position properly out-of-band per the OP suggestion.
Its not without an extra keyword - it would be a way to make streams processable by the for loop syntax analogously to if they were an iterator of futures, which would need to be awaited (possibly in the pattern).
This is ambiguous (unless we use a prefix syntax here and a postfix syntax in expressions) because you could be awaiting a future of a normal iterator.
Summary: I think await for reads naturally for the operation of async steam iteration. I get to this result by examining the 2x2 matrix of value production:
ninja edit: I forgot about the generator-for-loops proposal. This post thus ignores them. That said, I think it unlikely there would be reason to have a Generator return a Future, as a AsyncGenerator would have a poll point between the final item and the generator result anyway. (I’m in favor of a postfix await anyway, which then wouldn’t conflict here.)
Here’s an attempt to find some more structure in this feature, based on the construction of the Stream trait and the hypothetical design of async fn in traits. It suggests that, like generic const fn, we might apply async to traits, and then derives some syntax ideas from there:
I have a preference for postfix await. I balk at await <binding> because it’s completely different from await <expr> – the thing being awaited is completely different. The postfix syntax makes a lot of sense to me:
for <binding> await in <expr> -> loop { let <binding> = await ...
If we do add await? I would expect await? <expr> to work as well.
async Traitfeels right, as does try Trait even, especially now that const Trait has an established meaning (if not an accepted one).
I’d have to see them in action to be more certain how I really feel about them, though. const is “transparent” in that it doesn’t require extra syntax to move to/from it and “normal” Rust. This definitely feels right when the “effect” is “transparent”, but I could see it working for these non-“transparent” traits as well… EXCEPT:
This would also give async Drop as well, which is something people have expressed the desire for. An async Drop type would theoretically not be usable outside async context at all since it can’t be dropped synchronously. This sounds good, but it also introduces a “transparent” await point to a world that decided they want “noisy” await points (for good reason which we’ve discussed before and should not be rehashed on this thread).
Along those lines, what does try Drop mean? Drop really is a special trait.
The idea is strong. It may work for Rust. But I’m not sure right now.
I’d argue that both const and Drop are already special so that’s not a big deal.
const is special in the sense that it’s a kind of “inverse effect” the way ?Sized is an “inverse bound.” It limits what the function can do and makes it callable from more places, while try/async/generators do the reverse. The “non-const” effect is what we should be comparing them to, I think.
Drop is special in the sense that it’s mutually exclusive with Copy, and doesn’t participate in generics (everything is implicitly “drop-polymorphic” in its non-Copy type arguments). And the generic const fn RFC is looking like it will further extend this with T: Drop working on types that don’t literally impl Drop. So limiting Drop to non-try and non-async seems totally plausible.
(If we have more to discuss let’s do it off-thread and see if it goes anywhere?)
The discussion of graceful handling of iterators of results (as in lines()) brings to light again what is, IMO, the fundamental awkwardness of that construction. An iterator that reads from a file is not an iterator of independent Results, it’s an iterator that can fail completely, but the type signature conflates the two cases. Rather than trying to paper over the awkwardness there, I think we should look to represent that pattern more accurately, e.g. as previously discussed by @newpavlov.
async modifies things to make them create a future of something instead of evaluating to that thing directly - this is not what this syntax does. What this does is yield from the surrounding async item when the next item in the stream is not ready yet - exactly the same thing as what the await operator does to futures already.
Hmm, but this is in pattern position, where you expect things to be ‘reversed’. If you make await a pattern and allow let await foo = bar, consistency suggests that that should bind foo to some expression such that await foo == bar – as opposed to being equivalent to let foo = await bar. Same goes for ?.
Even if it’s not a pattern, this concern persists to some extent as long as it looks like one, i.e. if you have to put await immediately before a pattern, as in the case of for await foo in bar().
That said, I don’t have any better ideas for how they could be expressed as patterns. async could be seen as sort of like the opposite of await, but it isn’t really, and there’d be no equivalent for ?. On the other hand, it doesn’t seem all that inherently advantageous to make them patterns, and some of the proposed syntaxes for loop-specific constructs don’t have the “looks like a pattern” issue.
That's quite an interesting read. Considering T: async Trait seems like something we ought to explore as a means of getting code reuse and most bang-for-buck.
I think you get the duality of patterns & expressions right, but there's potentially a minor snag. Depending on how we view async { ... } and try { ... } (either 1. $ident $block or 2. $ident $expr) then the pattern form should try to mirror that. It is clear how you do that with 2. but less so with 1. This snag may be minor enough not to matter.
As @rpjohnst noted, try { ... } as a pattern is the dual to try { ... } as an expression.
Not only that. I think it is disadvantageous to make ? and await into patterns because:
It introduces side-effects into the currently side-effect free language of patterns (possibly modulo Drop).
By embedding ? and await into patterns, pattern matching (aside from guards) may now panic.
While I don't have a proof, as far as I know, pattern matching aside from guards and evaluating the scrutinee is not turing complete but embedding ? and await would make it turing complete by embedding .poll() and .into_result() into patterns.
Pattern aliases could be proposed now and would not have to take context into account, but if we do introduce ? and await as patterns, then we suddenly need async & try flavors of pattern aliases. I also don't know what effect this would have wrt. promotability (cc @eddyb and Never allow `const fn` calls in promoteds · Issue #19 · rust-lang/const-eval · GitHub).
I think one possibility wasn’t mentioned at all here. I use it in corona and that’s postfix (method call) on the stream. That turns the stream into an iterator. For corona it actually makes a lot of sense, as it is stack-full coroutine library, but I find it ergonomic and natural:
for elem in stream.iter_ok() {
}
If await is keyword, being able to await one future or await multiple times on getting the next item out of the stream should be possible.
I’m not really arguing for the possibility, just saying it seems to be overlooked.
Not necessarily! We just wouldn't allow them in pattern aliases (at least initially), and because pattern aliases are incredibly pure (pretty much as pure as types themselves), it would be sound to have some of their pattern "arguments" contain try/await patterns.
I mean... sure -- that sorta feels like a cop-out tho. Something about "aliases for patterns except for $these exceptions" does not seem great in terms of avoiding special cases.
They are even a total fragment, right? (making them "purer" than types which are an undecidable fragment if we forget about recursion limits...)
It doesn't feel like that to me, to be honest.
Just like ? is sugar for something in expressions, it could be sugar for a similar thing in pattern-matching, without ? being part of patterns, only of surface pattern syntax.
There is no pattern-matching going on in a pattern alias, so ? or await simply don't make sense there.
Similarly, if we start allowing nested guards in patterns, those wouldn't exist in pattern aliases.
They're code that runs during pattern-matching, but not "true patterns".
(We might want to include such features in pattern aliases eventually, but it'd have to be opt-in, e.g. async pattern Foo, and whatever signature shorthand we came up with for -> Result<T, E>, I forgot)
from_fn takes an impl FnMut() -> Option<T>, whereas an async function (one with the await superpower) doesn’t impl that trait, as it returns a Future representing the poll-able deferred computation.
