A final proposal for await syntax

I apologize for that, I tend to speak very bluntly and honestly about my opinions.

And it is very frustrating seeing people who refuse to respect the Rust team or their wishes, yet loudly demand that their wishes be respected.

I think I've said everything I needed to, so I will also disengage.

1 Like

So far the working group is not very easy to observe (much less join), and that thread doesn't shed much light. It's possible to discover their GitHub repo, but visibility is so low as to suggest that they're intentionally in some kind of "low-level stealth" mode, so I don't know whether I should even link to the GitHub.

That said, @skade, my personal opinion is that more visibility sooner would be a good thing, so I'd be grateful if you'd update the linked Internals thread announcing the WG with something, even if it's not much, just so that the people in that thread (including myself) don't feel like they're being ignored.

3 Likes

tradeoffs of postfix dot notations

I’ve been reading this thread. I want to try and leave a few thoughts. I’m going to write two comments in short succession. This first one aims to give a more detailed look at the various “postfix options” that are based on .. It begins by summarizing the key points around each option, and then gives me “personal narrative” that explains where I personally fall.

I want to emphasize that I am explicitly not considering future generalizations in this post. I’ll talk a bit about the reasoning in the next post, but the bottom line is that “you never know what may come”, so I’d like to reason about “what would happen if we had this syntax forever”. (But also, in the next post, with an eye towards future possibilities.)

Also, I apologize for the length of this post. If you prefer, it is also available on HackMd.

Go forth and experiment!

One thing I want to emphasize is that I think we are could use fewer comments in this thread and more lived experience. We’ve seen a lot of fragments of code, but those fragments were written using await!(). Now that .await syntax has landed, I would really like to see people go ahead and use it! Give yourself a little time to get used to it, but also pay attention to moments of confusion or surprise. If you can give concrete examples, that’d be helpful. This is the point where .await will feel the worst, since we don’t yet have syntax highlighting etc, so that should be a good torture test. I’d be quite interested to hear those reports.

Finally, people have brought up screen readers and accessibility. I do not understand how foo.await might be a problem there, given that we already have field and method syntax, but perhaps others can clarify that (I’d particularly like to hear from people who are actually using screen readers).

Feedback welcome

As always, I’m interested in getting feedback – but of a particular kind. I am looking for arguments (pros/cons) that you think are not included in this summary. If I see anything that seems new, I will add it to the summary – and, hey, maybe you’ll change my mind. =)

NB: I will update the HackMd version of this post, not this comment.

Executive summary

  • foo.await?
    • Pro: very lightweight
    • Con: easily confused for a field access, which conveys a “no side effects” intuition
  • foo.await()?
    • Pro: good analogy to blocking I/O; functions mean “something happens here”
    • Con: could mislead into thinking other methods could not block
    • Con: sort of strange to have a keyword with extra characters () “just because”
  • foo.await!()?
    • Pro: macros have always had ability to do surprising control flow
    • Con: very verbose – foo.await!()?
    • Con: not a macro, could never be
  • foo.await!?
    • Pro: fairly lightweight, clearly not a field
    • Con: could mislead into thinking ! is an operator (indeed, it is, for macros)
    • Con: no precedent for this sort of thing in Rust “syntactic tradition”
    • Amusing: means foo.await!? is a thing, for better or worse
      • But note that foo.await?! would not work – I actually made this typo a few times, this could be pretty annoying in practice. =)

Running example

To help make each syntax more real, I want to use this “running example” (expressed in naive prefix form). These ‘snippets’ are adapted from the await syntax repo:

let ids = (await interface_ids(service.clone(), interface_id))?;
 
let response = (await wlan.list_interfaces()).context("error listing ifaces")?;

.await

let ids = interface_ids(service.clone(), interface_id).await?;
 
let response = await wlan.list_interfaces().await.context("error listing ifaces")?;

The main concern about foo.await, from what I can tell, boils down to the fact that it will be confused for a field. This seems to offer the biggest possible “mismatch” between user expectations of the available options. In general, there is a sort of “spectrum” of side-effects one might consider from a given bit of syntax, and field accesses fall on the lowest possible side of that. Thus, making foo.await potentially block the current task and switch to another is potentially very surprising. (This is also an argument against progammable properties, which of course Rust doesn’t have.)

In Rust in particular, the “field vs operator” distinction can be significant in other ways. Method calls return temporaries, for example, so while &foo.bar returns a value that lives as long as foo, &foo.bar() creates a temporar that lives as long as the enclosing statement (typically). &foo.await would be the latter, which is unfortunate.

Clearly, syntax highlighting will help to avoid confusion. I think this is very important, but I can see counter arguments: e.g., there are contexts where it doesn’t apply: for example, when I write foo.await in markdown, I don’t get highlighting. Similarly, syntax highlighters often highlight keywords in incorrect contexts or miss things, so maybe users don’t trust this entirely.

It is worth asking how much trouble there will actually be a result of this confusion. I suspect that, most of the time, .await'ing a future will not, in fact, have visible side-effects on the data you have in hand, although it clearly can.

(In general, one of the big advantages of explicit await is that it lets you ignore async I/O most of the time, but when you need to care, you can. That is, when you need to audit for where side-effects occur, you can easily find them. This is similar to how ? lets you ignore errors a lot of the time, while making them visible. Field syntax doesn’t make it harder to audit, but it probably does cause those awaits to fade even further into the background than they otherwise might.)

.await()

let ids = interface_ids(service.clone(), interface_id).await()?;
 
let response = await wlan.list_interfaces().await().context("error listing ifaces")?;

One obvious answer to “fields accesses don’t have side effects” is to use a syntax that conveys the notion of side-effects. We could, for example, use await().

There is some danger here, though: this is not an ordinary method. It is an operator, despite looking like a method. It could lead to the “inverse” confusion – that is, that any method call may potentially block, instead of only this “special method” called await.

Similarly, writing .await() as a function may make people “think of it” as a method, and thus be surprised that they can’t use (e.g.) some fully qualified syntax like Await::await(foo) to invoke it without dot notation.

Ultimately, I think that the .await() syntax is leaning very hard on the “you can think of async I/O as if it were blocking” intution very hard. In other words, you can think of an async fn as running in its own thread, with .await() corresponding to a blocking operation where the scheduler may choose to run another thread. In that case, the analogy is basically perfect, even though the model is not at all what is happening. (I find that somewhat appealing, myself, but you can also see where it might be misleading.)

I personally find await() a bit surprising for other reasons: Rust has a kind of tradition of “bare keywords”, and somehow adding two extra parentheses “just because” feels surprising to me. But I think that’s a somewhat weak argument. (We do have unsafe { } blocks, which are somewhat similar.)

.await!()?

let ids = interface_ids(service.clone(), interface_id).await!()?;
 
let response = await wlan.list_interfaces().await!().context("error listing ifaces")?;

Certainly macros are expected to mean “this expands to unconventional things” – so we could imagine using !() to convey that in this case.

I think one very strong argument against this is just verbosity. Even with just .await, async I/O is already far less ergonomic than sync I/O – adding !() feels to me like a bridge too far.

Of course, we do use macros for a number of common purposes, such as println!, panic! and so forth. Indeed, those used to be keywords, but were made into macros to simplify the language and compiler. But here we don’t get that benefit: async/await is still a core language concept; it also must still be implemented in the compiler.

Separately, of course, it’d be nice to have general method macros at some point, but I don’t see that as really affecting this question very much. If we choose to add method macros, it would not conflict with the existing of a .await keyword in particular.

.await!?

let ids = interface_ids(service.clone(), interface_id).await!?;
 
let response = await wlan.list_interfaces().await!.context("error listing ifaces")?;

One answer to the concerns with !() question is to just use !. This builds on the "! means this may have unconventional control flow" intution, but without being overwhelming or looking like a “normal” macro.

It does mean that you would commonly see stuff like let x = foo.await!?, which is … kind of amusing. Still, it’s worth noting that foo.await?! would not work (it’s meaningless) – but normally when one combines ? and ! it is done like so “What on earth?!!!”". My fingers at least mistyped this quite a few times. =)

Still, while Ruby and Lisp have identifiers like foo? and bar!, there isn’t much precedent for this in Rust, where ! is used a an operator (though in ways that don’t, I don’t think, conflict with this syntax):

  • !foo of course;
  • but also $a!(..) in a macro, which invokes whatever macro name $a expands to.

My personal narrative

The previous sections were meant to be fairly dispassionate (if not truly “objective”). This section tries to explain why I personally lean towards foo.await (for the purposes of this comment, I am ignoring future compatibility).

I think the bottom line for me is that foo.await is the least intrusive syntax. When writing Async I/O code, I imagine one has to do a lot of awaiting, and most of the time you don’t want to think about it very much. I think people will learn quickly that it is, indeed, not a field, but rather an operator, and I think that syntax highlighting will help. I am also influenced by the fact that @cramertj, who has written a ton of actual async-await code in Rust, prefers .await.

I think that of the other options, I would choose foo.await() as a second choice. I don’t mind that it “looks like a method”, because I think that – in terms of its potential affects – await can be thought of as a “blocking method”. My main hesitation here is that, ultimately, a keyword suffices to remove ambiguity, so the the () is just there to “signal” to the user that this is not a field. But maybe that’s important enough to be worthwhile. (And, of course, there is no future generalization path here, but I’m trying to ignore that.)

I could live with foo.await! because it’s short, but ultimately it feels like more of a departure from our syntax to me.

I personally find foo.await!() to be too verbose and do not consider it a contender. I realize others disagree.

Feedback welcome

Let me just repeat what I said before:

As always, I’m interested in getting feedback – but of a particular kind. I am looking for arguments (pros/cons) that you think are not included in this summary. If I see anything that seems new, I will add it to the summary – and, hey, maybe you’ll change my mind. =)

NB: I will update the HackMd version of this post, not this comment.

30 Likes

In my previous comment, I looked at the "dot syntax" options and tried to discuss them. In this comment, I want to try and consider "the future". Again, I apologize for the length. If you prefer, this comment is also available on HackMd.

I've seen a few folks make an argument that roughly says:

We should just pick await foo for now -- then we might consider various generalizations going forward.

The reasoning is that there are a number of options that generalize await foo to permit smoother interaction with ?:

  • await?-like syntax
  • foo.await (and potentially foo.match)
  • a "pipeline operator" that could be applied to await (e.g., foo |> await or foo@await).

I definitely understand the desire to future-proof. But I am also wary of underweighting the present -- we have a lot of experience that shows that await foo is a poor fit for Rust's error handling strategy, for reasons that have been well documented by now.

In general, I think when we make decisions, we should consider the future, but also consider the very real possibility that a "temporary" solution winds up lasting a long time (or forever). This could easily mean that approximately 50% of the time, when people use await, they have to do something like (await foo)?.

Are parentheses a big deal? Of course, some people don't see that as a problem. And certainly many people accustomed to JS have told me that it's "not considered a big deal" to write (await foo.bar()).baz(). But I don't think that's the right analogy: I think the key point is that JavaScript uses exceptions for error-handling, and we don't, and that makes this a much bigger deal in Rust (and a much more frequent occurrence). I've found that using ? is so much nicer than using try!(..) ever was, and a significant part of that is that I do not have to "go back" and insert the try!( at the start of the line. Porting sync code to async will be much more painful if we have to go put ( at the start of the line. Is it going to make or break Rust? Of course not. But I think it's "distinctly suboptimal".

Future-proofing is not free. "Keeping your options open" sounds great, but it comes with real costs:

  • First of all, we have to spend time discussing and debating this design, instead of the polishing the implementation, or making streams work, or generators. (Or things entirely unrelated to futures -- this discussion has wound up "sucking in" folks like me, who would otherwise be hacking on other things.)
  • Second, if we make any change here, we're going to have to update documentation, books, stackoverflow answers, and so forth. It would be much easier if we can just write those docs once.
  • Finally, future proofing means that we are effectively locking ourselves into having "two ways to do it" in the future (unless we wind up sticking with the current choice, which I think is itself a problem). I am not convinced yet that we want two options.
    • Granted, one could imagine deprecating the "temporary choice" after a permanent one is found. But that implies even more transition costs.

How likely are we to wind up with .await anyway? I think this is a key question. At the end of this exploration, how likely is it that we will wind up with .await? If we will, then it seems better to just "cut to the chase"

Ultimately, I give it fairly high odds. There are basically two competitors: await? and a generalized pipeline operator. Let's look at them.

await? is distinctly disadvantaged by not interoperating with map_err and other such methods. Yes, you can implement those helpers for future types (though there may be coherence problems with that?), but at that point it feels like we're working awfully hard just to avoid postfix await. (I am not that worried about the "strangeness budget" here. The gap between await foo and foo.await is not that far, and there are clear reasons that the latter is a better fit for Rust.)

So what about pipelines? That's a bit harder to predict. But one thing I know for sure: I don't see us adding a generalized pipeline operator in the near term. That's a big syntactic bet for Rust, and I don't think we're at the point where that's a good strategy. So if we block on pipelines, I think that means we'll be using the "temporary" syntax likely for some time.

Worst case. So what's the worst case? If we adopt .await now and then decide we want to adopt a different "pipeline operator" in the future, what happens? I see a few possibilities:

  • maybe we find a way to build on the . operator, in which case everything is fine;
  • we could deprecate foo.await and add foo#await, despite the downsides;
  • we can just live with it
    • after all, method chains like foo.into_iter().map() are here to stay; is .await or .await() so different?
    • in any case, it's not obvious why a pipeline operator needs to apply to keywords anyway. After all, a pipeline expression a # b takes two values a and b and is equivalent to b(a), right? But await is not a value, it's an operator.

The TL;DR

I am sympathetic to arguments that we should "future proof", but I'm also cognizant of the costs it brings. When I look forward, I feel like some form of "dot postfix" syntax is the most likely winner. I am inclined to just make the decision and be done with it.

If we were going to future proof, I think I would be inclined to go with the await!(...) macro, and not the await foo syntax. This is because it is clearly a transition measure, and I think we're less likely to get stuck in a "local maximum".

I'm going to keep thinking about this one, though. I'm definitely curious to hear if people think I got something wrong in my analysis.

33 Likes

Minor typo: first await shouldn't be there.

3 Likes

(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)

Excellent write-up; thank you for taking the time for this.

First, some small feedback on form, rather than content: the question marks at the end of each bullet point confuse me. They’re not within the backticks, so I assume they don’t represent the ? operator. I’d recommend taking them out, unless the point is to show how each syntax looks in the “50% of the time” case, in which case they should be inside the backticks.

The only feedback I have on content is that there’s another aspect of .await() that I haven’t seen explained anywhere except in this blog post: it arguably could be implemented as a function, in which case it would be subject to Deref coercion. That blog post links to an issue that makes it unclear to me whether this is a pro, a con (following @cramertj’s comment), or both, but I do think it’s worth noting.

(@zackw, I think you’ll be gratified to read that blog post.)

1 Like

Since this is now copied here, let me copy eddyb's response from Reddit:

7 Likes

That makes sense; thank you.

2 Likes

I would have instead proposed it as #[await] fn or await fn in the first place, had I known the details. I don't think it completely negates the point that it could have an interface similar to that of a standard function. async fn itself are not functions even though they use fn in their type so an fn type able to influce the yield points of the calling coroutine (via forced inlining) does not seem out of the question. You'd need to ask eddyb for the specifics, but I interpret it as being mostly negative towards abusing extern abi for this cause not the approach as a whole. I'm aware of the bias though towards my own proposal.

2 Likes

Fwiw, re. markdown, this is what I see in VSCode:

markdown-highlighted

(The inverse does not hold, you don't see rust code in doc comments in rust code highlighted, but there's a ticket for that, Highlight code snippets in doc comments · Issue #393 · rust-lang/vscode-rust · GitHub)

For me, an important point about syntax highlighting is that it is often present when learning. You may not always have highlighting available, such as in git diff (I think this is the most notable occasion), but if you see await as highlighted in official documentation then the chances are good that you will understand it differently.

Another point re. await is that it is a verb whereas fields are typically nouns or adjectives. This makes .await stand out more while learning such that it is hard to confuse for a field.

A third point is that IDEs are quite adept in dealing with fields in terms of UX. Notably, VSCode combined with RLS gives you the type and visibility of field as well as documentation the user has provided:

field-hover-docs

It does not seem far fetched to me that RLS could offer such documentation on the await field itself. This would both give you the type of the value of extracted from the future as well as a note ("documentation") that this is a built in language construct. It seems to me that no other syntax could offer as good UX for IDEs.

A forth point I want to make around the ease of finding await points is that .await is about as easy as it gets in terms of using grep. You don't need to play any tricks with regexes to account for parenthesis or other contortions. .await should just work every time.

7 Likes

Thank you for taking the time to be so concise whilst staying so detailed. Everything you’ve said makes a lot of sense.

On the second post (related to the future) I’m curious what you think of the case where we introduce other postfix forms of keywords, such as the foo.match form which has been floated around. In this case would you consider it inconsistent that those keywords would have a postfix form in addition to a prefix form, whereas await would not?

1 Like

I'm going to get a bit meta, and call attention to Stroustrup's Rule:

  • For new features, people insist on LOUD explicit syntax.
  • For established features, people want terse notation.

As written, I see two ways to apply it here.

First, some proposed alternatives, like .await!(), are considerably louder than .await. Not that much longer in terms of number of characters, but a lot noisier looking due to the symbols, including an exclamation mark (a symbol that denotes literal loudness!) whose only purpose is to alert you that "something weird is going on". And certainly more explicit. Applying the rule suggests that experienced Rust users will come to accept and prefer the terser .await once the feature is "established" and they get used to it.

On the other hand, second, there are even terser options available, such as a postfix sigil or nothing at all (making await implicit). Now, terser is not always better, and Rust's philosophy includes making some things explicit that other languages would leave implicit, but it's still worth noting that .await may seem noisier in the future than it does today.

However, I'd also like to suggest a generalization of Stroustrup's Rule:

  • People have a negative reaction to the unfamiliar, especially if it seems to intrude on something they already know.

In other words, if I feel proficient in Rust's syntax, and suddenly there's this new thing that threatens to make my knowledge incomplete and outdated, it feels like:

And I'll want to build a big protective barrier around it in the form of loud explicit syntax, so at least I'll know when I can't rely on my existing proficiency. On the other hand, once I become proficient in the new thing as well, it doesn't seem like as big a deal, and it feels okay to have it "within the perimeter" of potentially conflatable syntax.

Even though .await is not actually hard to learn, per se, it still takes getting used to. As an experienced Rust user, I already have a mental model of the language, where everything feels logical and in its place. To create that model I had to spend time and effort learning new concepts one by one, but now that I've done so it all seems natural. Indeed, if I look back on the language, it feels sort of unnaturally "easy to learn", because what's easier to learn than something you already know? But if you then take that framework and stick a new concept in the middle, the concept comes off as illogical, weird, even hard to learn, simply by contrast.

But I'm not bringing this up just to psychoanalyze people with different opinions to mine... I'd also like to psychoanalyze people that agree with me. :wink: Something that's easier to derive from the generalization is that "people" includes people who are learning Rust. Since most parts of Rust's syntax are similar to other languages that they may already know, they would like to apply their existing proficiency to Rust, and seeing something unfamiliar like .await is a turn-off – in a way that's hard to fully empathize with if you are familiar with it. Something like .await may stand out less if it's just one of many unfamiliar things about Rust, as it would be for a learner, than if it's the only thing you're unfamiliar with... but small unfamiliarities add up. (And languages like Go have been incredibly successful by minimizing unfamiliarities.)

This is just a reformulation of the concept of the "strangeness budget", and it's nothing that hasn't already been considered and articulated multiple times. But I think it's useful to express it in terms of a two-sided cognitive bias:

  • On one hand, unfamiliarity can make something feel more problematic than it really is.
  • On the other, familiarity can make something feel less problematic than it really is (if your goal is making something friendly to newcomers).

Personally, I think .await is a good compromise when considering both of these principles. In particular, if you're a Rust expert, adding a new sigil may seem less weird than using .await, because both are equally new, but only .await intrudes on an existing point of knowledge (field access). But if you're new to Rust... I guarantee a sigil would at least be harder to learn, especially if you unexpectedly encounter it when reading code. Even if .await looks weird, it's pretty easy to guess what it does if you know await from other languages; sigils, on the other hand, are impossible to Google, and even once you learn what they do, they're harder to remember. For the same reason, I strongly suspect a sigil would also make the language feel more arcane overall, even if it avoids the "huh this isn't a field access" shock. (See also: Rust removing @ and ~ sigils in the pre-1.0 period.) The real advantage of a sigil, IMO, is just that it's terser; that's what we're giving up, and I'm okay with that.

But I also think it's important to keep in mind the volume of the opposition to .await: not only does it represent a faction of users in its own right, it serves as the best proxy, however imperfect, for the critical yet silent constituency of new users. And while it seems unlikely at this point that the team will change its decision, I think that opposition should continue to make itself heard.

16 Likes

That's true, but I think it's also important to point out that you get much of that benefit from syntax highlighting alone. I wouldn't want to assume that Rust learners have an IDE handy. IDE functionality isn't available in all contexts; some people don't want to use IDEs at all (or don't know how to get that functionality in their favorite editor); RLS is still in rough shape, and so on. But syntax highlighting is available almost everywhere.

9 Likes

Excellent write-up! Personally after reading it I become positively-neutral about .await, though I still see more appeal in universal pipelining.

I would like to comment on this bit:

To me universal pipelining is a more flexible construct, which transforms something prefix into postfix form. To make code more terse it employs set of "intuitive" rules (though some may call them "inconsistent" :man_shrugging:):

  • function/macro with one argument: a#f -> f(a), a#f! -> f!(a)
  • function/macro with several arguments: a#f(b, c) -> f(a, b, c), a#f!(b, c) -> f!(a, b, c)
  • keywords: a#await -> await a, a#yield -> yield a, a#match { .. } -> match a { .. }, a#if let pat { .. } -> if let pat = a { .. }, a#for x { .. } -> for x in a { .. } and others (though I am not sure about the latter two)

I think using pipelining sigil would be a bit less heavy on "strangeness budget", as it clearly speaks "this functionality new, you can't use intuition from other languages", though I understand the argument that since we don't know if pipelining in such form and prefix await will be ever added to the language, it makes sense to start with .await and then deprecate it (or simply leave it be) if such pipelining will be added at some point.

I’ve not yet seen suggested: un homenaje a la macro posteriormente:

future¡await
future¡await?

¡Feel the symmetry! Rust is tight with Unicode, no? And easy to type, at least for me, really. Everyone will certainly know: esto es especial! :slight_smile:

5 Likes

¡ is a fantastic suggestion for a sigil. Love it. :star_struck:

2 Likes

One thing I haven’t seen answered is how this plays in patterns. For example, is this valid syntax:

if let Some(value.await) = readAsync() {
  //...
}

Similarly, is this?

match wrappingEnum {
    Async(i.await) => process(i),
    Sync(i) => process(i),
}

WithoutBoats indicated the Postfix syntax might not play well with for loops. In that case would we special case for, and have it work differently form how await is used elsewhere? Or would we create something like ref for patterns where a prefix keyword on the left hand side can stand in for a Postfix on the right hand side?

If I saw future.await in like the above, I would guess that it would result in an odd compiler error because some other syntax is required. Where as I think await future conveys the expectation it should work in patterns and loops and both future.await() and future.await!() convey that it will not.

1 Like

No, await is only an operator that applies in an expression. There have been mentions of an await pattern related to @withoutboats posts on for loops, but there have been no serious proposals of adding them yet

See this later comment. (One of the ideas that I think I've seen is that if for loops can be generalised to streaming iterators with GATs then we can have a trivial transform of Stream<Item = T> -> StreamingIterator<Item: Future<Output = T>> and use that directly in a for loop).

3 Likes

One thing hasn’t been explicitly ruled out yet, AFAIK: having both prefix and some postfix await.

let result = await future; could just compile.

(await future) could cause an auto-fixable compiler warning, suggesting to use future.await instead.

await future?, i.e. prefix-await with parenthesis missing by mistake, could cause an auto-fixable compiler error, suggesting future.await?

Having prefix await would lower the barrier of entry for people coming from other languages while the warnings would gently nudge them into the right direction.That way .await wouldn’t break the unwritten and in fact non-existent rule that postfix-keywords have a prefix variant.

Another option would be the compiler recognizing await future as an error,with an auto-fixable future.await suggestion.

This isn’t meant as an endorsement. I just wanted to mention those options for the sake of completeness.

Personally, I’m perfectly fine with just future.await but anything mentioned above wouldn’t bother me, either.

I don’t think going with just prefix await and postponing .await until a more general .keyword has emerged would be a good idea.

2 Likes