A final proposal for await syntax

[jcsoo]

@mgeisler

My arguments have been that the .await field syntax breaks basic consistencies in the language. It introduces new flaws that the language didn’t have before. Therefore, I don’t see why it can be considered as a proposal in the same matter as the other three ideas.

I can totally empathize with someone that has the not-unreasonable mental model of .field_name as "field syntax" seeing this proposal as a strange inconsistency - after all, with that model, the only way to make sense of .await is as a magic field accessor, a one-off property getter with strange control flow.

But from what you know of the lang team, are they really the type to pick an ad-hoc solution just for the sake of being weird, or just to shoehorn in a feature? Everything I've seen from them suggests the complete opposite. The Rust way is to take a deeper look at the problem and solution space and come up with an innovative approach that is better than what's been done before.

In this case, the real underlying issue is that sometimes one prefers keywords in prefix position, and sometimes one prefers them in postfix position. If you have to pick only one variant forever, you end up penalizing the programming style that isn't favored. Also, there isn't much to go on as far as how to implement postfix keywords in any existing language.

The insight that was hinted at in the final proposal (and has been brought up a few times in related discussions over the last few weeks) is that the solution may be to loosen the restrictions on the . operator that we've inherited from other languages. Currently, keywords and macros aren't allowed as . operands, but in most cases it's pretty clear how they would behave if they were.

This requires changing one's mental model. One way is to add a new rule: .await is a special case. A different way is to start thinking of the . operator as a the "chain" operator and remembering what isn't allowed to be chained.

Yes, this adds to Rust's "wierdness budget", but this is doing it for the right reason: discovering a way to solve a problem that many programming languages have, but few have really tackled.

[scottmcm]

Thanks, boats, for another excellent post!

---

@The blog post

In this model, even the ? operator can be thought of as a modification on the period construct.

I'm now really amused by the idea that ? is actually just sugar for .ʔ

(For extra clarity, just in case: I'm definitely not actually proposing that the latter be made to work.)

---

@mgeisler

I guess both foo.await and foo.r#await would exist — and do different things

loop { x } and r#loop { x } also both exist and do different things.

This kind of thing is why raw identifier syntax exists, so I don't understand the argument that it's a problem.

---

@Cazadorro

you guys discussed things internally, and already moved passed ideas the community re-argued about even though they were dead in the water.

I'd like to point out that the paper writeup for the previous syntax thread explicitly said

For that reason, we have excluded sigil based syntaxes like @ from further consideration. If we adopt a postfix syntax, it will include the await keyword in some way.

That didn't stop sigil-based syntax from being proposed in the thread anyway.

If you have suggestions for better ways for teams to communicate the state of things, I encourage you to give feedback to the governance working group.

[BatmanAoD]

@devsnek

Something that other languages have done is to purposely split up suspend points so that it is immediately clear to the reader and writer what the control flow of the code is.

I have a friend (who unfortunately doesn't seem to be participating directly in these threads) who suggested that discouraging unlimited chaining of asynchronous operations would be a benefit of prefix syntax. Together with devsnek's point, that actually seems fairly reasonable to me.

@phaylon

I don’t have a source for this, but I’d assume an additional prefix option can have advantages for screen reader usage as well? Then you go into the expression knowing there’s an await point. Maybe someone with more experience here can give input as to how helpful prefix control flow is in that case.

I think that this is an extremely important consideration, and crucially, users of screen readers could easily be unrepresented in a discussion like this. (And, of course, it undermines the assumption that IDE features such as syntax highlighting can clarify otherwise-unclear syntax.)

[Screwtapello]

I’ve been very loosely following the discussion, but (like many other people) I was confused why expr.await was the most promising candidate, given the inconsistencies with field access and with other competing languages. However, the prospect of generalized expr.keyword support changes that entirely: if I can say expr.match {} and expr.if {} else {} are possible, then expr.await would be natural and consistent; and if match expr {} and if expr {} else {} are possible then await expr should also be possible, and would be consistent with other languages.

My only concern, then, is with the order of operations. If Rust stablises async/await support with the await expr syntax, but some obstacle prevents stabilising generalised expr.keyword syntax quickly, then Rust will be stuck with a suboptimal await syntax… but at least it will be a consistent and familiar one. On the other hand, if Rust stabilises async/await support with the expr.await syntax and some obstacle prevents stabilising generalised expr.keyword syntax (including the await expr back-formation), then Rust will be stuck with a quirky special-case syntax.

In order to get async/await stabilised and usable quickly, I think it makes sense to pick a bland and unadventurous placeholder syntax like await expr, and then work on stabilising the syntax we actually want with generalised expr.keyword.

(as another vote in support of generalised expr.keyword syntax, note that rust-analyzer already offers such things as completions and rewrites them to the stable syntax)

[lnicola]

@BatmanAoD

I have a friend (who unfortunately doesn’t seem to be participating directly in these threads) who suggested that discouraging unlimited chaining of asynchronous operations would be a benefit of prefix syntax.

I agree, as I don't really "get" the chaining argument. Take this basic code:

let mut frob = Frob::new(Widget::open()?)?;

Is this idiomatic, or is it better to split it over two lines?

let widget = Widget::open("")?;
let mut frob = Frob::new(widget)?;

I find the second one much nicer. And if we agree on that, why the push for

let mut frob = Frob::new(Widget::open().await?).await?

?

Perhaps I have a limited view, but I think (non-stream) async calls as being of two kinds:

• connect-style operations, like on a socket, HTTP connection, database, device discovery API, my Widget::open() example and so on, where the result of the future enables access to new functionality
• transfer operations, like message reads or writes

For the first kind, it's often desirable to reuse the "connection". Even when that's not necessary, it's not a bad idea to avoid chaining unrelated operations like establishing a connection and transferring some data.

For the second kind, these are often "one-shot" (wait for a timer, or open a connection, send a message, maybe wait for a reply) or involve looping (accept a new connection, spawn a handler for it, start over). For the former, chaining isn't bad (although it's totally fine to avoid it), and it's not applicable for the latter.

So I'll argue that for run-of-the-mill code (proxies and gateways, CRUD/REST services), chaining is not a requirement, and often not desirable. And when it can be used, it's often very limited; take a look at https://github.com/inejge/await-syntax/blob/postfix-field/bin/wlan/wlantool/src/main.rs and count the chains of three or more awaits in a row.

This means that there is no need to introducing new syntax solely to support chaining.

[comex]

@mgeisler

That’s a bit weird in itself and it becomes more weird and inconsistent when you realize that

foo.r#other_field == foo.other_field

holds for all other field names since the raw identifier quoting syntax is optional for normal fields. This is an inconsistency and it’s only there because existing syntax is being misused.

By the same logic:

You can define a function r#return:

fn r#return(_: i32) {}

And you can call it with r#return(x). However, return(x) also exists and does a different thing! This is weird and inconsistent, because r#foo(x) and foo(x) are equivalent for (almost) all other function names since the raw identifier quoting syntax is optional for normal names.

...In reality, it's not inconsistent. Keywords behave differently from identifiers; that's why they're keywords. The r#foo syntax to get an identifier named like a keyword is essentially a backwards compatibility hack; it mainly exists to handle the case where you're using a crate that was written in an older edition of Rust and defines an API with a name that has become a keyword in newer editions (for example, await became a keyword in Rust 2018). The entire point of the syntax is to suppress treating something as a keyword, so the fact that it behaves differently is not surprising. Although r#foo can also be used where foo is not a keyword, in which case it behaves the same as plain foo, that's not the intended use.

The only "inconsistency" is that await would be the first, and so far only, keyword that can appear after . in Rust. That would make it unique, but that's not the same as inconsistent. And as mentioned in the post, there is some thought of allowing other keywords in that position in the future, such as match. Also, there is plenty of precedent from other languages for allowing things other than field or method names after .. For example, C++ supports foo.operator+, foo.template bar, foo.*bar, and foo.MACRO (where MACRO could expand to anything). I'm not saying C++'s syntax is a good role model, but I think .await would be a good deal more principled than any of those syntaxes.

[mgeisler]

@jcsoo

I can totally empathize with someone that has the not-unreasonable mental model of .field_name as “field syntax” seeing this proposal as a strange inconsistency

Thank you — you sum it up very well.

@jcsoo

sometimes one prefers keywords in prefix position, and sometimes one prefers them in postfix position

Personally, I've never preferred a keyword in a postfix position since it isn't a thing. I've never wanted to write

foobar.return;

instead of the normal return foobar;. When should I use which? Similarly, I've never wanted to write

pred.if { ... }

or

elements.for { ... }

It's not even clear to me how the postfix for construct should work since the syntaxtical construct has two parts: a pattern expression and the iterator expression.

How far does this syntax go? Should

foobar.fn { ... }

be a thing? Why not?

Yes, having keywords in a postfix position would open up new possibilities and making this work for more keywords than just await would make the language consistent in this regard. Still, I believe we'll be opening up a whole new can of worms if we go into this direction.

@Screwtapello

In order to get async/await stabilised and usable quickly, I think it makes sense to pick a bland and unadventurous placeholder syntax like await expr , and then work on stabilising the syntax we actually want with generalised expr.keyword .

Exactly. A well-known and well-understood syntax will make the language more expressive with minimum risk. Even stabilizing the await!() macro would do that.

---

@scottmcm

I guess both foo.await and foo.r#await would exist — and do different things

loop { x } and r#loop { x } also both exist and do different things.

This kind of thing is why raw identifier syntax exists, so I don’t understand the argument that it’s a problem.

Yeah, true, raw identifiers allow you write fun stuff like if r#if { ... } and if r#false { ... } where r#false is true However, my point above was a little more quirky: .await would make it the first time that Rust puts a keyword into a position where we've so far only had identifiers. Since keywords can be used as identifiers with the raw identifier syntax, we have a new kind of clash here.

But I concede that it's probably not of great concern — raw identifiers were bound to cause this kind of clashes.

[repax]

@mgeisler

Personally, I’ve never preferred a keyword in a postfix position

Yeah, you've told us so many many times already...

@mgeisler

I’ve never wanted to write

foobar.return;

instead of the normal return foobar; . When should I use which?

return isn't eligible and you know that because you've read the proposal:

[...] potential future extensions in which some “expression-oriented” keywords (that is, those that evaluate to something other than ! or () ) can all be called in a “method-like” fashion.

[comex]

@mgeisler

Should

foobar.fn { ... }

be a thing? Why not?

It's pretty easy to distinguish this case because in fn foobar(), foobar is not an expression.

As for if and for... dunno. Personally, strangeness budget concerns would make me lean against them, but for better or worse elements.for is quite reminiscent of Ruby. [edit:] And .if would be similar to the C ternary operator.

[dan_t]

I couldn’t agree more. The whole await syntax discussion seemed a bit disproportionate, putting too much importance into the await syntax by trying to micro optimize every single use case.

[mgeisler]

@repax

return isn’t eligible and you know that because you’ve read the proposal:

[…] potential future extensions in which some “expression-oriented” keywords (that is, those that evaluate to something other than ! or () ) can all be called in a “method-like” fashion.

Sorry, I'm not a language expert and don't have a clear understanding of which keywords are "expression oriented". For me it seemed that return foo would very operate on foo, similar to how match foo { ... } operates on foo.

So in practice, I guess we're only talking about .if, .match, and .loop? Do we have other expression-oriented keywords?

I tried looking it up and found the ExpressionWithBlock term on Expressions - The Rust Reference — which seems to confirm the above list with the addition of the unsafe keyword.

---

@comex

It’s pretty easy to distinguish this case because in fn foobar() , foobar is not an expression.

Thanks, makes sense and matches the above.

[dan_t]

I really have a hard time following this argument.

Yes, consistency is a great value, but just allowing every keyword (match, if, …) to be postfix doesn’t quite remove the strangeness of a postfix keyword, it IMHO just opens the door for every different styles of writing code in Rust, which won’t quite increase the readability of Rust code.

[nilsbore]

Thanks for all the hard work on this issue, both with implementation and syntax issues!

There is one other alternative that I think would be reasonable, in addition to what has already been discussed. The reasoning for not having await be a postfix macro seems valid, since it is quite different from previous language “macros”. But what if a conservative keyword such as await {x} could be added to the language so that x.await!() could be thought of as implemented using that keyword (in a world where postfix macros are valid and the keyword doesn’t collide with the macro )?

[phil_opp]

@kornel made a really good point in another thread that the dot operator can already be seen as syntactic sugar for the (prefix) universal function calling syntax:

Idea: universal pipelining (a.k.a. making @await generic)

We have universal function calling syntax, so: instance.method() is equivalent to Type::method(instance). The D language (and to a degree, JS and Python too) goes as far as making all functions equally usable with both syntaxes. IMHO that’s pretty elegant. So I see . not as related to the type, but as a syntax sugar that moves the first function argument to the prefix position. Macros already pretend to be function-like. Therefore, making macro!(expr) and expr.macro!() equal would seem like …

Supporting both await expr and expr.await seems consistent with this model.

[Heliozoa]

I agree. I’m admittedly new to the discussion and a non-expert, but it seems to me that await expr is uncontroversial and almost inevitable; either as the initial syntax or after expr.await leads to expr.keyword, which leads to await expr for consistency’s sake. The proposal is solid, but I think it would be nice to have the more familiar alternative first and handle the general expr.keyword case in one go afterwards. Either way, thanks to the team for their work on Rust!

[kbknapp]

First off, thanks to @withoutboats and the whole lang team for trudging through this! The thoroughness with which this has been explored and debated is awe inspiring! Just to throw my two cents into the ring, I’d be most in favor of stabilizing the seemingly uncontroversial, if not conservative, await expr with the work towards expr.await including the more general expr.keyword.

That gives us time to explore the design space for expr.keyword in greater detail in case we discover some issue preventing stabilization we’re not left with (somewhat) quirky syntax for a particular subdomain.

Something that isn’t immediately clear to me is what expr.match, expr.if, and expr.loop (especially for) actually provide? With await I totally get it, it allows chaining and unwrapping Results. But all the other constructs include blocks (match expr { .. } -> expr.match { .. }). Based off that, it looks like the only thing it provides is just another way to write those constructs for aesthetics. In fact, if and for specifically actually seem a little more confusing with that variant (expr.if { .. } else { .. } and pat in expr.for { .. }). Is there something I’m missing?

I could maybe see a world in which these could be combined to form something like expr.match if expr { .. } where the match is guarded (currently if expr { match expr { .. } }). If that’s what we’re talking about, then I’d be very excited indeed.

[jcsoo]

@lnicola

I agree, as I don’t really “get” the chaining argument.

@mgeisler

Personally, I’ve never preferred a keyword in a postfix position since it isn’t a thing .

Going meta one level...

Rust is used in a wide range of applications by a wide variety of coders using a wide variety of programming styles. Because of this dynamic range (wider than any other mainstream language except perhaps for C++), it doesn't take much effort (if one is actively looking) to find code that is written in a style that seems alien.

Sometimes it's simply badly written code - this happens in every language. But often it's not; it may be using concepts that you aren't familiar with, techniques that you weren't taught or that were developed after your schooling, a style that comes from a different language or domain, or even something new developed as an experiment. Sometimes it's code written by amazing programmers that requires a different way of thinking about a problem, or even coding in general.

One of the things I admire most about the Rust community is its tolerance for this kind of diversity - acceptance of different types of programmers, different types of programming styles, and different types of applications. There is also a level of trust: trust that when there are multiple ways to do something, the programmer writing that code is the best placed to make the judgement of how to do it and even to make mistakes.

This doesn't mean that Rust should add every feature or every programming paradigm that comes along; things still need to be useful, interact well with other features, and not prone to misuse (in the "accidentally cause an incorrect program" sense).

But bringing up arguments of "I don't program that way" or "I'd do it this other way" aren't that helpful because they can easily be interpreted as "I don't program that way and nobody else should either" or "I'd do it this other way because the way you're doing it is wrong". This just ends up making people defensive.

[newpavlov]

I also think that await fut may be a better way forward to unblock async programming. While I firmly prefer a dedicated pipeline operator which will work with keywords, macros and functions, I can live with dot operator being used for postfix keywords and macros. But I will strongly dislike it, if .await will end up a sole postfix keyword, as it will be a really strange exception in otherwise rather consistent language. So if await fut will work in future, why not start with it? It will be less controversial and more familiar for those who learned async programming in other languages. Yes, (await fut)? is weird, but judging by Fuchsia code it’s not terrible, and there are ways to remedy parenthesis issue.

[huxi]

First of all, thanks a lot to the language team for the way this whole discussion was handled. I think you did an amazing job with the currently available tools (GitHub UX is less than ideal for issues with 500+ comments…). While there is always room for improvements you definitely shouldn’t feel bad about it.

My initial reaction to .await a few months ago was primarily disgust because it felt to me, at that time, like a terrible hack. My mind wasn’t ready to see it as anything else but a field access.

What I did not take into account back then was that we are not just talking about .await in particular but about .keyword in general and that keywords would be syntax-highlighted differently than fields anyway.

Merely mentioning the prospect of a potential future .match flipped a switch in my brain from “dot can mean two things” to “dot can actually mean three things” while changing my sentiment from “this is pretty weird. probably way too weird.” to “this is quite clever and beautiful”.

On its own, .await is a strange outlier. But a single other example turned it into “one of those dot keywords”, reducing the weirdness of both. They only appear weird or strange to us because they are something new not seen anywhere else - yet.

But objectively (and likely with hindsight a few months down the road) they won’t appear any stranger than how .method() resolves the first self argument.

I still think that Rust should also support the prefix variant, though.

The two variants cater to different coding styles (let rebinding vs. chaining) where neither is inherently better or worse than the other.

I think that "foo.await is sugar for (await foo)" would be a nice mnemonic while also preserving the common, well-known syntax from other languages.

Concerns about inconsistent code style could (and should) be addressed with configurable clippy rules.

[jcsoo]

@nilsbore

But what if a conservative keyword such as await {x} could be added to the language so that x.await!() could be thought of as implemented using that keyword (in a world where postfix macros are valid and the keyword doesn’t collide with the macro )?

I previously was in favor of postfix await!() but I've changed my mind:

• There would need to be a special case in the grammar to allow a macro with the same name as a keyword. Should users be able to define these macros for themselves, or can they only be defined in the standard library? Or is this special case limited to the await!() macro?
• If generalized postfix macros become a thing, the behavior of .await!() needs to match, but we don't know how we want generalized postfix macros to work yet. One big question is whether all macros can automatically be used in postfix position (like UFCS) or whether macros need to modified to explicitly take a $self parameter.
• If await!() can be used as a regular macro, you end up having both await!() and await {} for prefix use.
• If postfix keywords are implemented, you end up having both .await and .await!() for postfix use.

In contrast, .await appears to be compatible with any future postfix keyword proposal and doesn't require deciding up front how postfix macros would work.

I do agree with many posters that I would like to see await {} added at the same time as .await. We could then teach the basic concept of async / await using the prefix keyword (just as we can first teach error handling using match) and then teach postfix keyword as a useful variation, similar to ?.