It should be allowed. (assuming the method containing the code snippet you wrote is async).
If the issue is around dispatch, I don't see how the resolution is any different from a method with a generic in it's signature.
1 Like
My point is that it cannot be allowed. It is equivalent to this (using the current .await syntax):
foo.map(|x| x.await)
The above code snippet is invalid, it results in a compiler error, because you are using .await outside of an async fn.
The "asyncness" of a function is shallow, it does not extend into sub-functions or closures.
1 Like
It could. It's just more work to implement.
No, it really cannot. Deep coroutines have been discussed extensively and soundly rejected.
Deep coroutines aren't just some small little addition. They're a major redesign of the entire system, profoundly affecting all of Rust, and will likely require incredibly complex things like an effect system.
There's a reason Python and JavaScript don't have deep coroutines. In fact very very few languages have deep coroutines. Deep coroutines are equivalent to Scheme's call-with-current-continuation.
There is absolutely no way that deep coroutines will be accepted just to enable Await::await to work.
1 Like
I'm not talking about deep coroutines. Just look at the signature of 'map' above. It takes a function. This is by definition generic as the exact type won't be defined until the caller writes their code. So 'map' is going to get inlined into the caller. Once that happens, the await function is now happily in an async context and can be inlined. And everything works fine.
I get this totally cuts across layers in the compiler.
And I understand that inlining methods before validation messes with the neat order in which things are processed in the compiler, making it insanely difficult to implement. My point is this is at least theoretically possible without any runtime overhead, changing how stacks work, etc.
Interesting asside, that is exactly what macros do. So if we ever did get method style macros, this would come for "free". (Obviously it would not work with any existing code because it doesn't use method style macros)
There's a few major issues with that.
The first is that map will not get inlined. It will get monomorphized, which means it will create a function which is specialized to those particular types, but there's no guarantee that function will get inlined.
The second is that even if it was inlined, that is a performance optimization that should not affect behavior. I hope we agree that it would be bad if Await::await sometimes worked and sometimes didn't work depending on the whims of the compiler (or that the behavior differed in debug and release modes!).
The third issue is that the await is not actually in an async context... because it's in the context of the map, not the caller to map. Let's look again at the closure version (which is equivalent to Await::await):
foo.map(|x| x.await)
The .await is in the context of the closure (which is passed to map). So where should the .await yield to? To the outer async fn, or to the map? It can't yield to the outer async fn, because there's no guarantee that map will be inlined.
So it has to yield to the map. But then how can map call a closure which uses .await? The .await means that the closure needs to yield, which presumably means the closure needs to return a Future (I guess?).
But that means that the map somehow has to wait for that Future to finish... which means map now needs to be an async method too. And now the types are different, so you get compiler errors.
These are incredibly deep and difficult questions. This requires actual changes to the Rust language and the type system. Changes that need to be put into an RFC and agreed upon. It's not something that can just be solved with a bit of inlining compiler magic.
The argument for a trait method is that it's "less magic" than .await... but then you need to introduce massive amounts of magic to make it work. Much more magic than .await requires.
3 Likes
By the way, I’m working on drafting a “minimal trait ‘effects’” that would allow using impl (async Trait) where impl Trait is expected (though I’ve just realized that dyn Trait is another complicated thing to work out). It is not a simple feature to specify, though I think my formulation of it would be simple enough to use.
It’s definitely not something for this “finish what we started” year, but I’m aiming to have a nice flushed out description/proposal to publish at the end of Autumn or whenever people start taking about async Iterator/Stream again.
If people PM me I’m happy to share the work-in-progress blog post link. It’s still very in-progress, however.
1 Like
Fail to compile, hopefully with a very nice error message. But that would be far from specific to async_only fn. And the error could be nicer, I hope, because it is a specific failure for the moment.
// None of these are valid arguments to
// `map<F: FnOnce(T) -> ()>(self, f: F) -> …`
trait Await {
unsafe fn await(self);
// This one is typed as `FnOnce(Self) -> impl Future<()>`
// So valid to `map`, but with other return type as pointed out below.
async fn await(self);
// If stabilized as such, similar to above:
// This one as well as a generator, I'm not 100% sure
fn await(self) { yield (); }
}
So, even if we allow direct access to the method Await::await¹ then the concept that not everything denoted fn is also a valid FnOnce argument to map already exists in the language, I don't see any difficulty explaining it. And in fact, generators as currently in progress, have this property much more hidden instead of directly appearing as a keyword in-front of the fn itself to signal that this is a different kind of function.
¹The keyword would have to appear as if a call to that method but that doesn't mean it needs to be actually named await. So much should, imo, be signalled by a #[lang = "await"] on the trait instead.
That's equivalent to fn await(self) -> impl Future<()>, and, as such, is a perfectly valid parameter to Option::map.
1 Like
Oh, right. To a very different effect however than the other one, so it still shows how a keword before an fn definition must already be taught as modifying that definition to something other than a normal fn. Thanks for pointing this out.
But then that defeats the whole purpose of making it a trait method.
The reason for making it a trait method is to make it less magic, because it's just a normal method. So because it's a normal method, people don't need to learn anything new, and it doesn't require new syntax, etc.
But if you're now saying, "well, it's like a normal method, except not in these cases" people will wonder "why?" and then you have to explain that it's not really a method, it's actually special compiler magic.
Then people will ask "then why is it a method on a trait, if it's not actually a method on a trait?"
You cannot use the Await trait in bounds. You cannot use Await::await. So why is it being implemented as a trait method, if it never behaves like a trait method?
If the only valid thing you can do is foo.await(), then that's not a trait method, that's just new syntax!
Basically, you're introducing a huge amount of extra complexity for no benefit at all.
If you want foo.await() syntax, that's fine, but there's no need to introduce magic trait methods, it can just be new syntax.
Does it? Could you give an example of where such a restriction exists?
With generators, yield is a keyword, not a trait method, so the whole issue of "what does foo.map(yield) mean?" never comes up.
Using async does not create a special kind of function.
If you use async fn, it creates a normal function, which returns impl Future. You can use that function anywhere that a normal function can be used. It's not special or different.
The same is true for generators, which create a normal function which returns impl Iterator.
5 Likes
I don't disagree with this, but I want to point out that this argument applies word for word to syntax that looks like a field access, except more so, because it's completely impossible for .await to be a "real" field access, whereas (unless I've missed something, which is possible, it's hard to keep up with this discussion) .await() could be a real trait method in a hypothetical future version of the language (that has a bunch of features that may or may not ever get added because some of them might be more trouble than they're worth).
In other words: It seems to me as if everyone who is arguing that the await operation is too magical to be a method should like field access-like notation even less.
3 Likes
This was an alien perspective to me the first time I read it, because I was thinking synchronously. In a program that does synchronous I/O, any function call can potentially block, and I've written so much code in that environment that I could probably recite the list of POSIX system calls that can block from memory. It's normal to me for function calls to be suspension and cancellation points.
But you're right; in a program that does its I/O with async/await, you want to have confidence that none of the functions that you call from an async fn do anything that could cause synchronous I/O to happen. In fact, you may also want to have confidence that none of those functions will perform more than X microseconds worth of pure computation; anything longer than that needs to be pushed that off to a worker thread and wrapped in a future, just like I/O.
But I see this as an argument for an effect system or something, so that the compiler will detect it when async fn A calls library routine X calls library routine Y calls synchronous read and therefore your program is invalid. Avoiding the use of method notation for the await operation is never going to provide that level of assurance that mistakes will be noticed.
I am not super happy with ? for other reasons (related to my distaste for implicit return in general), but it's definitely not a function call, because it introduces a branch into the visible CFG. You have to be aware that every instance of ? is a point where the surrounding function might return to its caller, to understand what the surrounding function does. That's not the case with await.
1 Like
No, it really cannot. Because shallow coroutines are a fundamental part of the design of async/await (and indirectly, Rust in general, but that's a bigger conversation).
Theoretically, Rust could introduce a completely new system, like Scheme's call-with-current-continuation (or some sort of effect system), and then in that case we could implement await as a method.
But even then, that would be a new system, the old async/await system would still behave in the old way (because of backwards compatibility).
And it seems really weird to suggest that we implement a trait method now (even though it doesn't behave at all like a trait or a method), just because hypothetically 5 years from now we might get some fancy systems that make trait methods possible.
No, because .await is far less magic. It's just a keyword. It's not any more magic than return or break. It's easy to explain, and easy to understand.
But a trait method involves huge restrictions and/or massive compiler magic to support (and likely completely new features which haven't even been RFC'd yet). It's incredibly difficult to explain, especially with regard to all the restrictions.
I agree, effect systems are very cool. But you seem to be misunderstanding something: the Rust community has been waiting for async/await for literally years.
There is a strong pressure from important projects that want to use async/await, and there are deadlines to meet.
We're not going to wait for 5 years to debate a complex effect system and then implement async/await.
And that's assuming that an effect system would even be accepted (it likely wouldn't, due to the massive ramifications throughout the entire language).
That's not true. The .await operator inserts a loop and creates a branch in the state machine. A method cannot do that.
By its very nature it must affect the control flow, because of the fact that it needs to yield.
This is not just some implementation detail, it's a fundamental part of how async/await works (just like how return is a fundamental part of how ? works).
It's also not the same as synchronous blocking, because .await actually transforms the control flow of the async function (unlike synchronous blocking, which is just an ordinary function call, which doesn't affect the caller).
3 Likes
await could just as easily be a keyword inside an .await() construct. But if such a construct is adopted, there's less need to make it a keyword in the first place; keeping it a normal identifier would let it be subjected to ordinary name-resolution rules.
I'm not sure what you mean by this. If you mean that it would be a method that expands to compiler magic, I explained a few posts above why that doesn't work.
I mean, it would be a keyword like any other: its usage syntax would just coincide with method call syntax. The compiler would recognise the construct at parse time instead of after name resolution, and you wouldn’t be able to actually declare a method named await, because await would be a reserved word. Likewise you wouldn’t be able to declare variables, functions or other items named await. In other words, something similar to the sizeof keyword in C, which (most often) looks like a function call when used.
Sure, that’s perfectly valid. In that case it’s essentially just a stylistic debate between .await (which is inconsistent with field access) and .await() (which is inconsistent with methods).
Since they’re both inconsistent, there’s no objective benefit to one or the other.
The Rust team feels that the extra () is just unnecessary noise, so it’s better to go with the shorter option.
1 Like
I'll concede this point because it's not actually important to me. It's @HeroicKatora who seems to be arguing for await truly being implemented as a trait method. I'm probably never going to write an executor myself, so the existing Future API is sufficient to my needs.
What I care about is the notation and the surface implications of the notation. I should have written "It seems to me as if everyone who is arguing that the await operation is too magical to use method-like notation should like field access-like notation even less." (boldface words changed, everything else is the same).
I will stipulate that await is a keyword. I do not see how it follows that f.await is easier to explain than f.await(), particularly since there are a bunch of people saying that they find f.await confusingly similar to field access syntax.
My argument has always been that I think f.await() is more likely than f.await to indicate, to people who are reading async functions, the correct mental model for how it behaves. Namely: autoderef applies; it invokes code that depends on the concrete dynamic type of f; it may perform an arbitrarily lengthy operation before returning to the caller; and it does not introduce a branch into the surface control flow.
(It is also a suspension point and a cancellation point for the coroutine, as @RalfJung points out; this is communicated by the name await.)
It creates a branch in the state machine, but this is invisible in the surface control flow. Again, my argument has been that it is harder to debug async code if you have to think about the state machine; as evidence for this I point at what Promise-ful code looks like in JavaScript, particularly if you don't use .then and arrow functions. So I see a primary virtue of async/await as being that it hides the state machine, and I am in favor of notation that reinforces that hiding.
I don't think I ever said we should do that? I just meant to indicate that, in the absence of an effect system, the programmer has to be aware that there's a bunch of existing things written with function or method call notation, that perform synchronous I/O or unbounded computation, and therefore must not be used from an async context. So I don't see "we don't have an effect system the await operation can block" as a valid argument for "the await operation shouldn't look like a method call".
1 Like