Pre-Pre-RFC: async methods & bounding async fns

The types are named after the function but they are all written in PascalCase and the function name is written in snake_case, e.g. map() and Map. There is a clear difference between the two names that helps distinguish between type and function and the Rust naming conventions are followed to the letter.

You're right. Thanks for the code example!

1 Like

I don't have time to write a carefully-crafted post right now, so please forgive me jotting down a bunch of scattered thoughts!

Relationship between async fn and fn -> impl Future

I very much agree with @MajorBreakfast.

The "synchronous preamble" pattern (using -> impl Future with an inner async block) is likely to play a nontrivial role in async programming in Rust. For that reason alone, I think it's very important that you're able to move completely smoothly between async fn and fn -> impl Future.

One way we can make this smoother is to introduce a special lifetime, 'input, that bounds all input lifetimes (even if elided). Thus we'd have:

async fn foo(x: &Type1, y: &Type2) -> i32 { 
    /* body */ 
}

// equivalent to

fn foo(x: &Type1, y: &Type2) -> impl Future<Output = i32> + 'input { 
    async { /* body */ }
}

// equivalent to 

fn foo<'a, 'b>(x: &'a Type1, y: &'b Type2) -> impl Future<Output = i32> + 'a + 'b { 
    async { /* body */ }
}

As a general rule, treating a new feature as pure sugar for existing features has a ton of benefits. It means there are no truly new feature interactions to consider, since each interaction is explained by the desugaring. It improves interoperability, since you can use the "explicit" and sugary forms interchangeably. And it provides a multi-level understanding of the feature, allowing you to think about it as a first-class concept most of the time, but "drop down" into the desugaring when helpful. (Closures are probably the best example of this today, although you cannot implement the closure traits in stable Rust yet.)

One other thing I would add: I think it's important that you usually be able to use the most idiomatic (i.e. sugary) signature style when defining methods within traits. That means using e.g. impl Iterator and async fn. As things stand, there are limitations that prevent these forms from being "fully expressive" when using in trait definitions, but we should work to remove those limitations. Which brings us to the next topic...

Naming anonymous types

I understand @rpjohnst's perspective that async fn can be viewed as a type introduction form, but I have several reservations to treating it specially in this regard:

  • Every fn is a type introduction form, and over time I expect we will expose that fact more explicitly. Given that we want to be able to view an async fn as an fn, that means that async fn really introduces two types: the type of the function, and the type of the future it returns. I think it's important that any design for naming the type take this into account, to avoid confusion in the future (if and when we expose the fn type more directly).

  • My perspective has always been that we will have abstract type for truly tricky cases, but in the common case just using impl Trait should suffice. (In particular, as I mention above, I want to be able to unreservedly recommend people use impl Trait in trait definitions, rather than having idioms vary by context). My assumption was that we'd have some ergonomic means of projecting the entire return type of a function, which would suffice for async fn and the majority of impl Trait returns (given that they usually encompass the entire return type); I think it's fine to recommend abstract type only for the nested case (Vec<impl Trait> which I expect to be less common. This has already been discussed earlier on the thread with Output -- I think we should continue exploring this space to find something that works for both async fn and impl Trait more generally.

  • More concretely, we could e.g. consider outputof(path_to_fn) as shorthand for <typeof(path_to_fn) as FnOnce>::Output. I think there's a lot of design space here.

Edit: as per @MajorBreakfast's earlier points, this should probably be TypeOf and OutputOf, just like we have Self as a keyword in the "type namespace".

async at the end

Finally, I want to take up @lordan's suggestion of writing async closer to the return type; I think it's worth some serious thought. Some observations:

  • We could probably enable a slightly more precedented syntax this way: fn foo(&self) -> async i32 + Send. That is, we could allow + to be used within an async return type to impose additional bounds.

  • Writing it in the return type has a stronger connotation that this is a function returning an async value (sorry @rpjohnst!) And in particular, the jump from that to putting async immediately in the body feels very natural to me (and I wonder whether something similar holds for try).

  • If we did go this route, I think we'd want to strongly consider calling the trait Async rather than Future, for obvious reasons :slight_smile:

@withoutboats, I'm sure you've thought about this space before -- what are the pros/cons as you see them?

9 Likes

fn() -> async i32 being “future-wrapping” has a nice similarity to proposed “result-wrapping” fn() -> i32 throws Error :+1:

I dislike outputof and prefer <typeof(Foo::bar)>::Output, on grounds of orthogonality.

If the more explicit <typeof(Foo::bar) as FnOnce<_>>::Output form works, then I’d suggest maybe having OutputOf!(Foo::bar) that expands to that form (not the shorter one, just in case other Output associated types exist), for convenience.

3 Likes

The 'input lifetime is a brilliant idea! I think it's even possible to add this in a 100% non-breaking way to Rust 2015. It can be defined by default and an explicitly defined lifetime called "input" simply shadows it (plus there should be a warning).

Edit: With this notation rustdoc could automatically transform async fn signatures to fn my_fn() -> impl Future<Output=..> + 'input signatures. That way it'd be consistent!

You're right. This hasn't been considered yet and it should. Here are my thoughts:

  • Pro: Putting this in return position emphasizes that async functions are just another kind of function. They obviously don't execute their main work synchronously, but functions that return an async block don't do that as well and they also have no special marker in their signature.
  • Con: The effect of the async keyword isn't limited to the return type. It actually transforms the whole function into the future type. Putting it in front, like in the RFC, suggests that the whole function is being transformed. This is what's actually happening: The return type is replaced with the outer return type and the function body is transformed into a state machine.

I don't like it ^^' Here are some comments about this:

Another observation: We would get the same situation if future was keyword. Then, we'd have future blocks, future closures and future functions and the same problem that we can't name a variable "future" because it's a keyword.

1 Like

Thanks for writing up your view of our convo, @aturon. I’m going to avoid the interchangeability issue because I don’t know how to express my thoughts just yet, and just respond to the two other comments:

Naming anonymous type

I’m comfortable with outputof(path) as the syntax as long as we have an easier form for introducing the bound at the def site, such as async(Send). That is, I think

where T: Trait, outputof(T::method): Send

Is a very comprehensible way to write this bound. (I don’t have a strong opinion about casing and delimeters etc for outputof and typeof).

I do prefer outputof to typeof()::Output. The fact that outputof is sugar for a special case of typeof is not really a compelling argument against it in my opinion. Its clearly the most common use for typeof & I think its much easier to understand the first time you see it.

async at the end

None of this issue of adding the Send bound was unknown when we made the decisions about where async goes, and I don’t think anything here suggests we change it out. I’d prefer not to re-open this question.

Overally, I don’t think this:

fn foo() -> async i32 + Send

Is any more clear than this:

async(Send) fn foo() -> i32

Both are unique, ungeneralizable syntactic sugars. I’d make these additional notes:

  • The asyncness is a property of the function, not the return type. There is no such type as async i32. When we’ve had special return types, we’ve tended to want to generalize them to other type positions: ! and impl Trait are good examples.
  • async i32 + Send is not similar to how type syntax works otherwise. In addition to being a totally unique way of saying a type is bounded (shouldn’t it be async i32: Send?), its grammatically ambiguous with non-dyn trait types, which I think are still allowed just linted against, and in general we have not allowed types to be followed by + in our grammar.
  • I want to limit the deviation from the standard async function syntax as much as possible. Other languages put the async before the function declaration & this is what users expect (its called “async functions” after all, not “async return types”). This is a big deal to me that I think is often overlooked in these discussions: we can only deviate so much from the ‘platonic norm’ before it starts to lose some of the advantages of async/await.
  • I don’t want to re-open the name of the Future trait; as we’ve discussed before, I think there’s a lot of built up understanding and “brand” around that trait name that I wouldn’t want to give up.

I did consider this alternative:

async fn foo() -> i32: Send { }

I prefered async(Send) for a couple of reasons:

  1. Putting the bound by the async connects it to the future type, rather than the interior return type.
  2. I’m worried that the : Send could get lost pretty easily when scanning (-> Vec<T>: Send where T: Send for example).
  3. I don’t want ambiguity issues if we ever want to use a : as a part of our type name grammar someday.
11 Likes

Thanks @withoutboats! Your points against "async at the end" are very strong, and I’m convinced.

4 Likes

What about impl Trait in non-trait definitions? Even for bare fns (or inherent methods) returning impl Trait, it is common enough to want to, say, store a function’s output in a struct, and for the vast majority of functions, I don’t think there’s any way to be confident that a caller will never want to do that. But I don’t want to end up in a situation where it’s considered unidiomatic to use impl Trait return types anywhere in a public API, since IMO that would kind of cripple the feature.

I guess an alternative is to allow impl Trait in struct field types and elsewhere, so that in most places where you’d currently want to name a fn’s return type, you could just write impl Trait instead. In other words, impl Trait starts to act even more like its own “type”. Is that what you were thinking of? If so, that’s fine, and IMO better than encouraging use of typeof for this purpose. But I think it’s worth noting that that’s implicitly included in the vision.

2 Likes

About "output of":

  • I didn't notice it in the case of typeof because I see it everyday in JavaScript, but the two words concatenated together look a bit weird.
  • I'd suggest to make it lowercase because it itself is not a type, it just returns a type
  • It could be a compiler built-in: output_of!()

My preferred syntax is still fn_name<..>::Output where Output refers to the associated type of the FnOnce trait. It doesn't get any shorter.

You said that a while back. I've also never questioned why turbofish has the extra :: in there. Is it to avoid parsing ambiguities? Why do you think that that the turbofish operator should be used there?

1 Like

This would be committing to "a function's name is also the name of its type," which seems okay to me but also kind of a big deal? For one, that's currently not allowed- you need the <T as Trait>::Output syntax, so enabling that would be new. Second, error messages currently print out a function's type as fn(A, B) -> C {foo}, which seems important because it's close enough to the function pointer type syntax that you could guess that the coercion exists.

(Also, there's something about the current grammar not allowing turbofish in the middle of a path, but I'm not totally clear on when that applies.)

1 Like

Yes, it's to avoid parsing ambiguities between <> used for generics and as the less-than/greater-than operators. It shows up wherever generics are used in expression context.

2 Likes

Fair point, though... A big part of the need for declval in C++ was the lack of another way to ask “what type does this function return, given these argument types?”. There’s now std::result_of for that, but its implementation just uses declval. In Rust, on the other hand, you can theoretically just project to a Fn* trait's Output associated type.

I say theoretically, because currently there's a pretty silly obstacle to doing so. You can write

<MyFuncTy as FnOnce<(u32,)>>::Output

but that depends on unstable (and ugly) syntax. You can't write

<MyFuncTy as FnOnce(u32)>::Output

because writing FnOnce(u32) implies a return type of (). Nor does using an underscore work:

<MyFuncTy as FnOnce(u32) -> _>::Output

I wonder if that could be made to work backwards-compatibly. That is, when the return type is _, instead of desugaring to FnOnce<(Args,), Output=_>, just desugar to FnOnce<(Args,)>.

How about 'in? That's already reserved because in is a keyword, and it's more succinct.

2 Likes

Attractive because it's so short. I like it.

'params would also work nicely. However, the shorter the better, I suppose.

1 Like

Here’s a code example: I hope I didn’t get anything wrong. I was obviously only able to check the currrently working stuff

Common: Setup code shared between all following code examples

struct MyStruct<T> { x: T }
trait MyTrait<T> { type Item; fn foo(&self) -> &T; }
impl<T> MyTrait<T> for MyStruct<T> { type Item = Box<T>; fn foo(&self) -> &T { &self.x } }

let my_struct = MyStruct { x: 42 };

Expression context: Referring to method without <.. as Trait>

let _ = <MyStruct<i32> as MyTrait<i32>>::foo(&my_struct); // Current
let _ = MyStruct<i32>::foo(&my_struct); // Proposed
let _ = my_struct.foo(); // Current: This already works for method calls

Type context: Referring to associated type without <.. as Trait>

let _ : <MyStruct<i32> as MyTrait<i32>>::Item = Box::new(123); // Current
let _ : MyStruct<i32>::Item = Box::new(123); // Proposed

Bad future

  • We need <.. as Trait> to refer to a method or an associated type
  • We need <typeof foo> to refer to the type of a function
let _ : <<typeof <MyStruct<i32> as MyTrait<i32>>::foo> as FnOnce(&MyStruct<i32>) -> &i32>::Output = my_struct.foo();

Looks like C++ template programming ^^’

(See the scrollbar? Did you use it? :smile:)

output_of

let _ : output_of!(<MyStruct<i32> as MyTrait<i32>>::foo) = my_struct.foo();
let _ : output_of!(MyStruct<i32>::foo) = my_struct.foo(); // Without `<.. as Trait>`

Question: I wonder: What’s actually inside of output_of?

  • It’s not actually an expression because it’s incomplete
  • Type also makes no sense because typeof and outputof are introduced to avoid making functions types

Edit: Answer by @withoutboats

Good future

let _ : MyStruct<i32>::foo::Output = my_struct.foo();

We can read from left to right, yay!

5 Likes

Perfect. Then we should be fine in <typeof ..> or with Type::Output without turbofish because those are type contexts. Edit: I can't decide what context is in inside <typeof ..> (I even said so in my previous post ^^')

BTW I see that nested comparisons without parenthesis lead to an error. Maybe turbofish can be removed in the future without making the parser type aware? Who knows... Doesn't matter for this discussion, though.

1 Like

If functions are addressable as a type to query their <... as FnOnce>::Output, it would make sense to take a $:ty match.

However, this isn't possible even with $($:tt)+ matches and unboxed_closures in user code:

error[E0575]: expected associated type, found method `MyTrait::foo`
  --> src/main.rs:13:23
   |
13 |     let _ : OutputOf!(<MyStruct<u32> as MyTrait<u32>>::foo)
   |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not a associated type

I'd like to note though that if we get <..> elision for generic arguments, OutputOf! can almost be implemented purely in library code:

macro_rules! OutputOf {
    ($($path:tt)+) => (<$($path)+ as FnOnce<..>>::Output);
}

Of course, if it becomes a compiler built-in macro to work around the unstable nature of the Fn* traits, then you could handwave a bit and say it takes a "path to a method" (which seems the compiler already knows about, to say that it isn't an associated type).


Sorry, I didn't see that you mentioned type directly. I'm too tired to reorganize this on my phone but the last paragraph is still directly relevant, it just needs the earlier bits for context.

I've also mixed up what works and what doesn't a bit at this point so I may have missed something obvious, that's why I'm leaning on actually putting things on play.

Assorted notes:

  • I like 'in. Short and sweet as @MajorBreakfast said.

  • We can improve upon <MyFuncTy as FnOnce<(u32,)>>::Output with a type alias like so:

    type Out<F, Args> = <F as FnOnce<Args>>::Output;
    
    Out<MyFuncTy, (u32,)>
    

    Using Out will ensure that the bound FnOnce<Args> is satisfied but it is not checked on the type alias itself right now.

  • I second @withoutboats’s view on -> async T. Some more notes…

  • async is an effect, just like unsafe, const (anti-effect / restriction) and try (assuming we have try…) are. Whatever we do, I think it is important that there be consistency between effects so that we don’t have (more) of an unprincipled mess. That is, if we write:

    fn foo() -> async i32 { .. }
    

    Then you should also write:

    fn foo() -> unsafe i32 { .. }
    fn foo() -> const i32 { .. }
    ...
    

    However, the ship has sailed on unsafe fn.

  • One could imagine the following syntaxes (which I’m not suggesting…)

    fn foo() -> async(T, Bound) { .. }
    fn foo() -> async(T) { .. }
    
  • Some criticisms of async(Bound) fn foo() -> i32:

    1. Bound is too front and center, especially if Bound is long.
    2. It appears before quantification of type variables and where, adding yet another syntax for bounds.
  • With the introduction of 'in, maybe consider the following instead:

    trait Async<T> = Future<Output = T>
    
    async fn foo(x: &Type1, y: &Type2) -> impl 'in + Async<i32> { .. }
    

    or even:

    trait Async<T, trait B = 'static> = Future<Output = T> + B;
                   // ^--- This depends on quantification over bounds.
                   // Can't do that right now; but it can be added later
                   // backwards compatibly. EDIT: Actually, probably not..
    
    async fn foo(x: &Type1, y: &Type2) -> impl Async<Type3, 'in> { .. }
    
    async fn foo(x: u32, y: u32) -> impl Async<u32> { .. }
    
    async fn foo(x: u32, y: u32) -> impl Async<u32, Send> { .. }
    

    This seems to work better in conjunction with proposed schemes for try fn:

    try fn foo() -> Result<T, E> { .. }  // return type is mentioned as above!
    

I prefer <MyFuncTy as FnOnce(u32,)>::Output because that's more direct. However this syntax should only be required in cases where it is necessary to resolve ambiguity, i.e. next to never: For the type of a function there'd be never any ambiguity (because functions can't implement arbitrary traits) and for most other types ambiguities are really rare (we hardly ever need to use this syntax to call methods after all). So the short version MyFuncTy::Output (or MyType::MyAssocType in general) should almost always work.

^^' complicated (too indirect for my taste). But the idea is good. It's essentially specifying the outer return type:

async fn foo(x: &Type1, y: &Type2) -> impl Future<Output=i32> + 'in { .. }

The RFC mentions 3 reasons against specifying the outer return type:

  • Lifetimes: Can be considered solved by 'in. Also, if the inner to outer type conversion like the RFC defines it isn't considered too drastic, a simple shortcut that makes + 'in optional for async functions is certainly also possible!
  • Polymorphic return not required: The rationale is that it's always an impl Future, so there's no need to specify it. While this is somewhat true, we've now discovered serious downsides to this which the RFC does not mention. See next section.
  • Documentation: With point 1 addressed I consider it a rather weak argument

Upsides to specifying the outer return type after all:

  • Bounds can be specified as usual. No weird async(Send) fn required
  • Seamless integration with abstract types
  • It makes async functions more similar to regular functions
  • async then only affects the body, not the signature
    • async (like mut in fn foo(mut a: i32)) can simply be omitted in Rustdoc.
    • This makes the signatures of functions using the initialization pattern and functions defined via async fn identical.

It's a bit late now that the RFC is merged. But nobody had the 'in lifetime idea when we discussed the RFC. Going with the outer return type has advantages and we should talk about it again now that we have these insights.

3 Likes

So the idea was to shorten Future<Output = T> to Async<T> which becomes somewhat more ergonomic if you write it often. However, this trait alias can be added in user code, so you can make it as short as you wish if you repeat it often. If it turns out to be a common problem after years of experimentation, the alias can be added to the standard library.

One additional benefit of -> T instead of -> impl Future<Output = T> is that it mixes well with ideas about async-polymorphism, i.e: if you could write:

?async fn foo() -> T { .. }

Then you can let the caller decide whether the return type is a future, or just a normal T. This could enable us to write a library that is polymorphic over async-ness and let the client application decide. Writing -> impl Future<Output = T> out explicitly takes that option away.

Of course, writing:

?async(Send) fn foo() -> T { .. }

would make no sense.

Sure. However, we're talking about saving 8 characters, so I guess not ^^' Edit: If you want something short: How about a (user defined) fut!(T) macro? Will the proposed type trait aliases even be able to convert between a generic param and an associated type?

Hmm. Reminds me of the mut-polymorphism I've read about discussed somewhere long ago that was ultimately discarded. As a result we now have for example Iter and IterMut. Both are nice ideas, but IMO too ambitious to be practical. Some downsides:

  • It means that we need to specify whether we want to use the sync or the async version in some cases. Type inference will sometimes not be able to tell
  • my_fn::Output is impossible. At least not if there's no way to specify whether we want the sync or the async version.
  • A lot of async code requires a different implementation than the sync code. So the ?async syntax will often be not applicable
  • You mention ?async(Send). I agree. It's weird. Same goes for the awaits of course. They would also not affect the sync case
  • Traits are currently the only source of polymorphism that Rust has that I'm aware of (not counting macros). There's beauty to that. Edit: Corrected by @Centril

My conclusion: I don't think that it makes sense to add async polymorphism to the language.

2 Likes