A final proposal for await syntax

At last week’s language team meeting, we discussed the await syntax again and reached a tentative consensus about a timeline for a final decision as well as a concrete proposal for what syntax we think we are most likely to accept. I describe the timeline, the syntax, and the reasoning behind this decision, in this blog post:

Here is the tl;dr, but please read the longer post if you want to understand the reasoning:

  1. We hope to stabilize async/await in the 1.37 release, which branches on July 4.
  2. We will make a final decision about the await operator syntax at the May 23 meeting.
  3. We are inclined to adopt the postfix “dot await” syntax, future.await.
69 Likes

Given the tradeoffs explained in this post, I think the preference expressed here is sensible.

On the other hand, I am left wondering if the boring and slightly ugly (await something)? should also be on the alternative table – especially if/since it now leaves the door open for the more ambitious expression-oriented-keyword syntax later as a path forward. Postfix field syntax does seem to present a “flight forward”, intentionally embracing the weirdness to make the oddity more obvious.

1 Like

Note, that the dot in EXPR.await is technically unnecessary, so the proposal is future-compatible with making . optional and migrating to just a post-fix keyword EXPR await.

EDIT: The dot will indeed be necessary for disambiguation if other keywords like match get a post-fix form.

Was the UPS proposal discussed? I would appreciate some comments from the lang team. If @ and # are considered too noisy we also have ~ (fut~await?) or even two-symbol sigils. And I am not sure about using highlighting system as an argument in syntax selection.

3 Likes

I considered this for some time, but ultimately I decided that it didn't make sense. Basically I think that, when it comes to await, postfix is going to be the most convenient and predominant form of using await. The arguments here seem pretty strong. So if we only have one form, we want that form.

But there is a pre-existing problem that "prefix match" is sometimes a poor fit. This comes up most notably with try { foo } blocks, which yield a Result that sometimes wants to be matched. Plus there is obviously precedent with "prefix await".

Therefore, it might be nice to "square the circle" and support .keyword, as the post described. However, this is not a slam dunk, and I wouldn't want to block on it. For example, it obviously introduces a "more than one way to do it" question that we would want to think through. It's also not the sort of extension that is clearly on the roadmap for this year.

In short, I think it is an interesting future possibility, but it's really just a possibility, and in the meantime I think postfix await is a superior choice.

EDIT: Re-reading the post I felt it made it sound like this was my choice alone. To be clear, this is just meant to be my "personal opinion" for why I would prefer not to select prefix await.

26 Likes

We did consider this. But many members of the team (myself included) feel that since practically every await operator will return a result, having to use around parens to apply the ? operator is an immediate problem we want to solve now.

And like niko, I think the method syntax for keywords is just an interesting possibility, not something that's made inevitable by this decision.

This is discussed in terms of using the . operator for this, which (unlike you, I know) the team finds most agreeable if we were to introduce this concept. We are just generally not in favor of introducing new meaning to punctuation characters if it isn't strictly necessary; we find these worse than introducing additional meanings to dot. @ and # were examples; this reasoning applies to all of the options.

15 Likes

Moderator note: I'd like to quote something from @withoutboats's post for emphasis:

I firmly hope that everyone will extend to us good faith that we have considered this decision at length and very seriously, and have discussed and thought about all options.

Comments that don't uphold this simple discussion parameter will be removed. Discussion of this feature is difficult enough as it is. Accusing the language team of ignoring community opinion on this issue is explicitly not a discussion in good faith. Please find a way to more constructively voice your opinions.

As always, if anyone has any concerns about these minimal standards for decorum, do not bring them up here. Instead, email the mods: rust-mods@rust-lang.org

30 Likes

I understand and agree with the reasoning why DOTawait is the only viable postfix syntax at the moment.

Personally I always strongly prefered a postfix variant, since I do think that lack of one seriously impedes chainability and flow of code.

I also appreciate the difficulty of this decision.

BUT:

Some form of native pipelining support as discussed in a related thread and mentioned in the post with a postfix match etc would be a great solution, but I’m very doubtful that DOT is the right separator for this.

It puts a very serious strain on Rust’s weirdness budget and the mental overhead of distinguishing very distinct concepts of field/method access and special keyword control flow like await/match/etc is considerable . While maybe acceptable if this stays await only, it will make (mentally) parsing code more strenuous and confusing, even with syntax highlighting. Some form of separator like Elixir’s |> would be a much nicer choice IMO.

Also, I reckon that the implications for a generalized postfix DOT KEYWORD syntax have not been fully explored, but Rust will be stuck with this syntax forever. Especially if a generalized postfix keyword solution is on the horizon at some point, which would almost become inevitable in the future unless await will be the weird single special case in the language.

DOTawait is something I can live with, and the community will adapt. But I think, like @djc mentioned, the conservative prefix (await X)? is much smarter choice right now. Even though I was considerably against this previously.

With a potential generic postfix on the horizon, the possibility for making it nicer is there and can be introduced in a way that is consistent for the whole language. await will work just fine as a prefix, even if more awkard, with the benefit of not rushing to a major decision about control flow that can, in the long term, affect how a considerable amount of code is written.

EDIT:

After some reflection, I’m now very sceptical of introducing DOTawait without a consensus that this is the right form for generalized pipelining/postfix keyword application. Which is a much larger discussion.

22 Likes

BTW, it might be valuable if someone could make a pseudo-code attempt at explaining what the await syntax is actually expands to and why it’s not expressible in Rust and/or a macro. I feel like I have an intuitive understanding of this, but clearly the details matter a great deal.

If that is the case, since it would be less ambiguous, was :: considered instead of dot?

3 Likes

Just wonder if anyone would make a font that includes a special ligature for the “.await” combination - i’ll definitely use it.

5 Likes

I personally believe that every point against every posfix syntax is valid :

  • the field access looking operator seems completely wrong since it is a complex operation, the exact opposite of a field access
  • method access is equally wrong since it is technically not a function
  • postfix macro is far less misleading, but it is not a real macro either
  • the postfix keyword (with space) would be something unprecedented in any language I know

I personally think composing is better but I still wonder if I want to accept a Frankenstein syntax for this.

The postfix symbol was ruled out early because “people don’t like sigils” while it seems to me it defeat all the problems mentioned . The comparison with the try! macro was used again in this blog post. The symbol was the right solution for a postfix syntax at that time. I’d like to know why it is not an acceptable solution anymore.

10 Likes

Thanks for the thorough map of the team’s thoughts!

It’s not clear to me why it matters if the user thinks it’s a macro (but it actually isn’t, instead expanding to a first class secret inner construct). This seems like an implementation detail.

From the blog post it seems like the original macro solution works from a language design point of view, but is broken from a diagnostics point of view – it’s tricky to have it return decent errors, and it’s tricky to forbid bare yield from being used in async after stabilization. These seem to be implementation details to me: it would be fine if it were internally implemented as not-a-real-macro. We’ve considered replacing macros with compiler magic in the past – at one point I was working on an implementation of derive(Copy) that worked via MIR for optimal codegen. It would work the same, it was just the easiest implementation choice for decent codegen. This seems to be a similar thing, it could be a proc macro, but diagnostics would be terrible.

The only issue is bare yield – if await is supposed to be looked at as a (proc) macro, there is no way to implement your own custom await since yield isn’t allowed bare. If this is really an issue we can downgrade the error to a deny warning so people totally can implement a postfix await macro, it just has terrible diagnostics. We have precedent for “compiler/stdlib implements something better than you can but with the same semantics”, see Iterator’s rustc_on_unimplemented or the implementation of Vec. Both use unstable things for providing a better experience.

But with that being the only problem (since diagnostics are an implementation detail) to me it feels much less surprising to have a macro that really can’t be implemented as one as opposed to a syntax conflicting with field access. Like I said, it’s not clear to me why it matters that it can’t be implemented as a macro: very few people will ever try. “It’s almost a macro, but not quite” isn’t too bad; we have a bunch of those already. You’re right that those are mostly not-macros for specific reasons that don’t match with the reason here, but at this stage we’re very much at a point where most users won’t notice at all – I doubt that many users of the syntax will look into the implementation, and I doubt the ones who do will notice that it’s a different kind of built-in macro not seen before.

On the other hand, the confusion with field syntax is immediately apparent. You’re right that a user that knows anything about its semantics will realize it’s not a real field access, but for users who don’t this will be immensely confusing. I wrote code for quite a while before I encountered await, it was quite foreign to me when I did, but at least it was obvious that it was something I need to look up. This isn’t true for dot await, it looks like a field access and those new to the concept may gloss over it.

Even for those people who understand what it does, it seems to me that “a new kind of field syntax” will cross the minds of far more users than “a new kind of builtin macro”, since most users don’t know or care that builtin macros (as distinguished from other macros) are a thing, it’s an implementation detail to them.

The generalization to match/etc is pretty interesting, though. That might lead to a less confusing mental model (especially if async is syntax highlighted well), but it’s also another large bite into the strangeness budget. (Then again, so are generalized postfix macros)

Edit: I later realized that there’s another solution that addresses all the points and completely decouples await from yield: We support prefix await, and postfix await (as a macro or a field access) is treated as sugar over that. In such a case postfix await is just a normal postfix macro (which we don’t have but folks have wanted for a while) that wraps around prefix await. When you need chaining there’s a convenient macro to use. When you don’t, the familiar syntax exists. Seems very win-win. This also works well for .await if we generalize other keywords to postfix (.match, etc), though I personally prefer the macro.

40 Likes

As someone who originally was for the foo.await!() and foo@await syntaxes, and vehemently against the field access syntax, this post was originally very concerning. However this post did an excellent job of explaining your position. You did a very good job of covering all major positions not brought up by the team. I now feel that macro syntax is not the way to go, and that a keyword is necessary.

I originally was afraid of the syntax highlighting story, but I forgot that rust employs await as a strict keyword, thus making it trivial for syntax highlighting to work correctly. I also was afraid of potential compiler confusion by not using a special sigil instead of merely dot, but in combination with the opinion of the language team that this syntax should be feasible:

foo.bar(..).baz(..).match {
    Variant1 => { ... }
    Variant2(quux) => { ... }
}

and the strict keyword restriction, the sigil holds no value to me now, and I now agree, it is just noise. Part of the justification of the sigil was to promote a general pipelining syntax. What is more, this doesn't preclude a potential UMCS feature if needed, and would fit right in.

I now feel after this post that .field syntax is the best possible syntax for rust. I do think, however, there is a lot to discuss about the communication break-down that happened between the language team and the community previous to this.

There's the obvious, the fact that you guys discussed things internally, and already moved passed ideas the community re-argued about even though they were dead in the water. I think that is a rather simple lesson to learn from, log even more ideas you previously considered and rejected.

Then there's the massive confusion on whether await was implemented as a macro, a function, a compiler implemented thingy, and that still exists. I've seen people tell me "this is not a macro" and then say it is more a of a function when it clearly wasn't. We literally have an await macro, and that makes things even more confusing. We have an issue about the error messages of the await macro, and a claim that this why it couldn't be implemented as a macro, and that it was the only reason, it wasn't. You might have wanted to have a post on this in its own right.

The least straight forward communication failure to fix is how opaque the issues are around rusts tool-set, and how the compiler, well, actually works. With out a zeitgeist of community understanding of how the rust compiler actually works, we can't know the potential parsing issues, we either have to take the language teams word for it that "X isn't going to be a parsing issue", apply our knowledge from what other compilers do, or be an expert on rusts compiler in our own right. What is more concerning, is that the community has no collective idea about the issues of the proposals they themselves brought forth. Solutions that just won't work for one reason or another will have runaway support. With things like space await syntax, we just aren't going to know anything like this:

  • Postfix space expressions can cause problems with compiler recovery when the user omits a necessary semicolon (that is, they can result in the compiler giving much less helpful error messages); we’d like to avoid them without a compelling motivation.

with out compiler knowledge. And what is more, we are now going to hold your word as scripture. We will likely apply this logic to where it doesn't belong at all, causing more confusion. Knowledge about compiler restrictions and how hard it is to do, or modify something in the compiler needs to be more common, and that just isn't happening, even with the compiler team videos. You don't need to have everyone understand it, but you do need to have more people be able to give educated arguments on why you can or can't do something because of the compiler.

9 Likes

I was never a particular fan of .await, but I didn’t significantly prefer any of the other postfix options much more. I think its much more important that we bring this discussion to a close, and .await is a reasonable outcome. We’ll get used to it, and perhaps love it.

The generalization of .match (presumably also .return, .break, .if, .while(?) etc ) is intriguing. Conceptually it reuses .'s method call meaning of “take my left expression and put it as the first expression on the right”, but applied to keywords. However if we do end up doing this, then it means that await expr will have to become valid syntax to be consistent with the rest.

23 Likes

When looking at

I actually prefer an await!( ... ) macro over an .await field. Both would have magical properties, but: macros are already used to extend the language with custom syntax and with constructs that cannot be expressed in normal functions. It’s normal that “weird” or “unusual” stuff takes place inside a macro, so using one for await means that the user will only have to take a very small leap of faith in trusting the compiler.

If people don’t like the idea of a macro that cannot be implemented by the user, well then the obviously correct choice is to use a keyword. That’s what keywords are for :slight_smile:

Therefore, please don’t be so fixated on this “error handling problem” that you’re willing to abandon core language design principles — keywords are exactly what we use in case there is a construct that is built into the language.

Trying to generalize .await by saying we could also have .match, .for and other “keyword fields” seems like a very poor way to justify this huge step away into uncharted and weird territory. It will lead to a more complicated and confusing language since there are now two ways of using every keyword.

17 Likes

Thank you for explaining the decision.

7 Likes

Thanks. This is a good and thorough post which mirrors many of my own thoughts. This series of events will definitely be a primary case study for the meta working group.

12 Likes

I feel a possible argument for using a different sigil not mentioned in the post is that a different sigil would allow a universal postfix function syntax, i.e. you could have:

foo$await
f(foo, 42) == foo$f(42)

Not sure though if this kind of thing is desirable since it adds a redundant way of doing the same thing.

1 Like

A scenario just occurred to me, which I haven’t seen discussed before. I believe .await makes it harder to refactor code. Imagine your line becomes too long or too complicated and that you would like to extract some of the values into local variables. So you start with

return some_expression.await + other_expression.await;

You now refactor this into

let foo = some_expression.await;
let bar = other_expression.await;
return foo + bar;

Seems simple enough. However, there is now an observable difference between the above program and this program:

let bar = other_expression.await;
let foo = some_expression.await;
return foo + bar;

This feels like a big deviation from how Rust programs normally function and I bet that inexperienced Rust developers wont catch this on a first reading since it looks like simple field accesses.

10 Likes