Pre-Pre-RFC: async methods & bounding async fns

[MajorBreakfast]

I think async functions should be considered functions. For each async function there is of course a 1 to 1 relationship to the future type it defines, but it itself should be considered a function, and not directly its future type. In particular, to emphasize this, an async function and its type should not share a common name (although technically possible). With the initialization pattern we can actually separate the function part and future type:

fn foo<'a>(arg1: &'a str, arg2: &str) -> impl Future<Output = usize> + 'a {
    // Sync work
    async move { /* Async work */ }
}

Here the future type is defined by the async block. It should be possible to implement a function that does no synchronous work and just returns an async block in such a way that it is indistinguishable from an asynchronous function defined via async fn. Or in other words converting between the two ways should be 100% API compatible. Use cases:

• Convert async function to fn returning async block to add a tiny bit of synchronous initialization work
• Multiple asynchronous functions in the same library: Some are defined via async fn, some are not because they use the initialization pattern. The user should not have to care about which is which when referring to the types (Edit: This means it must be possible to refer to the type no matter whether it’s from an async fn or from an async block that is returned from an fn)

---

How about: Extending the async notation so that it lets us name the abstract type?

abstract type Foo: Future<Output = i32>;
async(Foo) fn foo() -> i32
/* desugars to: */
abstract type Foo: Future<Output = i32>;
fn foo() -> Foo

I’m proposing this, because I think that long-term ::Output or typeof make sense to look into, but we might want an easier to implement near term solution for this problem as well.

[MajorBreakfast]

Another thing that just came to mind:

trait Trait {
     fn foo(&self) -> impl Future<Output = i32>;
}

Could we implement this trait with the async fn notation in the current proposal? Note there’s no associated type called foo. If async fn automatically desugared to declaring the function plus setting the associated type, then it would expect the associated type to be declared. Consequently we’d get a "type foo not a member of trait Trait" error. The notation from my previous comment does not have this problem because it does not wire up the associated type unless it is told to do so.

[MajorBreakfast]

I just realized something else: It’s about tuple structs vs async functions

You’d never want to define a tuple struct in a trait. All you’d ever want to do is to define it outside of a trait and assign the associated type of the trait the tuple struct’s type.

OTOH defining async methods will be common. And they act exactly like normal methods: They can borrow self, we can define the function arguments, etc.

Tuple structs and async functions are very different in how they work: An async function defines its return type. A tuple struct defines its constructor. So, it’s function first vs type first.

(Edit: Sorry for putting all this in three separate posts. I didn’t think of it all at once and IMO three posts are better than making major changes to an existing post)

[Latrasis]

One thing I would like to ask about this RFC, isn’t it premature to at this point to decide on sugar for Futures until we have garnered enough data for 0.3 on stable?

[rpjohnst]

@MajorBreakfast

I think async functions should be considered functions.

Certainly. But they should also be considered type definitions, because that's also what they are.

The proposed syntax does not prevent you from doing what you want:

abstract type foo<'a>: Future<Output = usize> + 'a;
fn foo<'a>(arg1: &'a str, arg2: &str) -> foo<'a> {
    // sync work
    async move { /* async work */ }
}

As far as impl Trait in traits goes, @withoutboats has discussed the tradeoffs there earlier in this thread. Trying to match async fn syntax with -> impl Trait syntax is tricky and not something people should be doing regularly.

For that matter, using impl Trait in public APIs is not something people should be doing either, so that's not a signature people should be trying to match anyway. The reverse, implementing an async fn trait method with an -> impl Future, is less tricky and works fine with the proposed desugaring.

@MajorBreakfast

You’d never want to define a tuple struct in a trait.

You certainly would. There has been discussion of syntax like this, for defining an associated type inline:

impl Trait for T {
    struct A {
        x: i32,
        y: i32,
    }
}

And you'd want this even more for async fns. One of the primary blockers for the ecosystem to switch from hand-written poll functions to async fns is the ability to name the futures, and thus implement methods on them, store them in structs, etc. The pattern of "define an (associated) future type and its constructor" is all over the place, so making the async fn syntax define both of them makes a lot of sense.

Giving them the same name is even following an established pattern---take a look at the types returned from future and iterator combinators. They are all named after the function that returns them. Switching that pattern to impl Trait would lose the ability to name the return type; switching that pattern to "async fns (and potentially generator functions) also declare the name of their return type" solves that problem.

[MajorBreakfast]

@rpjohnst

take a look at the types returned from future and iterator combinators. They are all named after the function that returns them

The types are named after the function but they are all written in PascalCase and the function name is written in snake_case, e.g. map() and Map. There is a clear difference between the two names that helps distinguish between type and function and the Rust naming conventions are followed to the letter.

@rpjohnst

The proposed syntax does not prevent you from doing what you want:

You're right. Thanks for the code example!

[aturon]

I don't have time to write a carefully-crafted post right now, so please forgive me jotting down a bunch of scattered thoughts!

Relationship between async fn and fn -> impl Future

@MajorBreakfast

I think async functions should be considered functions. For each async function there is of course a 1 to 1 relationship to the future type it defines, but it itself should be considered a function, and not directly its future type.

@MajorBreakfast

It should be possible to implement a function that does no synchronous work and just returns an async block in such a way that it is indistinguishable from an asynchronous function defined via async fn . Or in other words converting between the two ways should be 100% API compatible.

I very much agree with @MajorBreakfast.

The "synchronous preamble" pattern (using -> impl Future with an inner async block) is likely to play a nontrivial role in async programming in Rust. For that reason alone, I think it's very important that you're able to move completely smoothly between async fn and fn -> impl Future.

One way we can make this smoother is to introduce a special lifetime, 'input, that bounds all input lifetimes (even if elided). Thus we'd have:

async fn foo(x: &Type1, y: &Type2) -> i32 { 
    /* body */ 
}

// equivalent to

fn foo(x: &Type1, y: &Type2) -> impl Future<Output = i32> + 'input { 
    async { /* body */ }
}

// equivalent to 

fn foo<'a, 'b>(x: &'a Type1, y: &'b Type2) -> impl Future<Output = i32> + 'a + 'b { 
    async { /* body */ }
}

As a general rule, treating a new feature as pure sugar for existing features has a ton of benefits. It means there are no truly new feature interactions to consider, since each interaction is explained by the desugaring. It improves interoperability, since you can use the "explicit" and sugary forms interchangeably. And it provides a multi-level understanding of the feature, allowing you to think about it as a first-class concept most of the time, but "drop down" into the desugaring when helpful. (Closures are probably the best example of this today, although you cannot implement the closure traits in stable Rust yet.)

One other thing I would add: I think it's important that you usually be able to use the most idiomatic (i.e. sugary) signature style when defining methods within traits. That means using e.g. impl Iterator and async fn. As things stand, there are limitations that prevent these forms from being "fully expressive" when using in trait definitions, but we should work to remove those limitations. Which brings us to the next topic...

Naming anonymous types

I understand @rpjohnst's perspective that async fn can be viewed as a type introduction form, but I have several reservations to treating it specially in this regard:

• Every fn is a type introduction form, and over time I expect we will expose that fact more explicitly. Given that we want to be able to view an async fn as an fn, that means that async fn really introduces two types: the type of the function, and the type of the future it returns. I think it's important that any design for naming the type take this into account, to avoid confusion in the future (if and when we expose the fn type more directly).

• My perspective has always been that we will have abstract type for truly tricky cases, but in the common case just using impl Trait should suffice. (In particular, as I mention above, I want to be able to unreservedly recommend people use impl Trait in trait definitions, rather than having idioms vary by context). My assumption was that we'd have some ergonomic means of projecting the entire return type of a function, which would suffice for async fn and the majority of impl Trait returns (given that they usually encompass the entire return type); I think it's fine to recommend abstract type only for the nested case (Vec<impl Trait> which I expect to be less common. This has already been discussed earlier on the thread with Output -- I think we should continue exploring this space to find something that works for both async fn and impl Trait more generally.

• More concretely, we could e.g. consider outputof(path_to_fn) as shorthand for <typeof(path_to_fn) as FnOnce>::Output. I think there's a lot of design space here.

Edit: as per @MajorBreakfast's earlier points, this should probably be TypeOf and OutputOf, just like we have Self as a keyword in the "type namespace".

async at the end

Finally, I want to take up @lordan's suggestion of writing async closer to the return type; I think it's worth some serious thought. Some observations:

• We could probably enable a slightly more precedented syntax this way: fn foo(&self) -> async i32 + Send. That is, we could allow + to be used within an async return type to impose additional bounds.

• Writing it in the return type has a stronger connotation that this is a function returning an async value (sorry @rpjohnst!) And in particular, the jump from that to putting async immediately in the body feels very natural to me (and I wonder whether something similar holds for try).

• If we did go this route, I think we'd want to strongly consider calling the trait Async rather than Future, for obvious reasons

@withoutboats, I'm sure you've thought about this space before -- what are the pros/cons as you see them?

[kornel]

fn() -> async i32 being “future-wrapping” has a nice similarity to proposed “result-wrapping” fn() -> i32 throws Error

[eddyb]

I dislike outputof and prefer <typeof(Foo::bar)>::Output, on grounds of orthogonality.

If the more explicit <typeof(Foo::bar) as FnOnce<_>>::Output form works, then I’d suggest maybe having OutputOf!(Foo::bar) that expands to that form (not the shorter one, just in case other Output associated types exist), for convenience.

[MajorBreakfast]

@aturon

One way we can make this smoother is to introduce a special lifetime, 'input , that bounds all input lifetimes (even if elided).

The 'input lifetime is a brilliant idea! I think it's even possible to add this in a 100% non-breaking way to Rust 2015. It can be defined by default and an explicitly defined lifetime called "input" simply shadows it (plus there should be a warning).

Edit: With this notation rustdoc could automatically transform async fn signatures to fn my_fn() -> impl Future<Output=..> + 'input signatures. That way it'd be consistent!

@aturon

writing async closer to the return type

You're right. This hasn't been considered yet and it should. Here are my thoughts:

• Pro: Putting this in return position emphasizes that async functions are just another kind of function. They obviously don't execute their main work synchronously, but functions that return an async block don't do that as well and they also have no special marker in their signature.
• Con: The effect of the async keyword isn't limited to the return type. It actually transforms the whole function into the future type. Putting it in front, like in the RFC, suggests that the whole function is being transformed. This is what's actually happening: The return type is replaced with the outer return type and the function body is transformed into a state machine.

@aturon

If we did go this route, I think we’d want to strongly consider calling the trait Async rather than Future , for obvious reasons

I don't like it ^^' Here are some comments about this:

• My comment why I don't like Async as trait name
• @glaebhoerl's comment and @thomaseizinger's comment that say that "async" is not a noun and using it inside a variable name therefore looks odd
• @withoutboats detailed comment about sticking with "Future"

Another observation: We would get the same situation if future was keyword. Then, we'd have future blocks, future closures and future functions and the same problem that we can't name a variable "future" because it's a keyword.

[withoutboats]

Thanks for writing up your view of our convo, @aturon. I’m going to avoid the interchangeability issue because I don’t know how to express my thoughts just yet, and just respond to the two other comments:

Naming anonymous type

I’m comfortable with outputof(path) as the syntax as long as we have an easier form for introducing the bound at the def site, such as async(Send). That is, I think

where T: Trait, outputof(T::method): Send

Is a very comprehensible way to write this bound. (I don’t have a strong opinion about casing and delimeters etc for outputof and typeof).

I do prefer outputof to typeof()::Output. The fact that outputof is sugar for a special case of typeof is not really a compelling argument against it in my opinion. Its clearly the most common use for typeof & I think its much easier to understand the first time you see it.

async at the end

None of this issue of adding the Send bound was unknown when we made the decisions about where async goes, and I don’t think anything here suggests we change it out. I’d prefer not to re-open this question.

Overally, I don’t think this:

fn foo() -> async i32 + Send

Is any more clear than this:

async(Send) fn foo() -> i32

Both are unique, ungeneralizable syntactic sugars. I’d make these additional notes:

• The asyncness is a property of the function, not the return type. There is no such type as async i32. When we’ve had special return types, we’ve tended to want to generalize them to other type positions: ! and impl Trait are good examples.
• async i32 + Send is not similar to how type syntax works otherwise. In addition to being a totally unique way of saying a type is bounded (shouldn’t it be async i32: Send?), its grammatically ambiguous with non-dyn trait types, which I think are still allowed just linted against, and in general we have not allowed types to be followed by + in our grammar.
• I want to limit the deviation from the standard async function syntax as much as possible. Other languages put the async before the function declaration & this is what users expect (its called “async functions” after all, not “async return types”). This is a big deal to me that I think is often overlooked in these discussions: we can only deviate so much from the ‘platonic norm’ before it starts to lose some of the advantages of async/await.
• I don’t want to re-open the name of the Future trait; as we’ve discussed before, I think there’s a lot of built up understanding and “brand” around that trait name that I wouldn’t want to give up.

I did consider this alternative:

async fn foo() -> i32: Send { }

I prefered async(Send) for a couple of reasons:

1. Putting the bound by the async connects it to the future type, rather than the interior return type.
2. I’m worried that the : Send could get lost pretty easily when scanning (-> Vec<T>: Send where T: Send for example).
3. I don’t want ambiguity issues if we ever want to use a : as a part of our type name grammar someday.

[aturon]

Thanks @withoutboats! Your points against "async at the end" are very strong, and I’m convinced.

[comex]

@withoutboats

This means impl Trait in trait definitions would be uncommon in public APIs unless the API author was confident there’d be no reason to constrain on that hidden type.

What about impl Trait in non-trait definitions? Even for bare fns (or inherent methods) returning impl Trait, it is common enough to want to, say, store a function’s output in a struct, and for the vast majority of functions, I don’t think there’s any way to be confident that a caller will never want to do that. But I don’t want to end up in a situation where it’s considered unidiomatic to use impl Trait return types anywhere in a public API, since IMO that would kind of cripple the feature.

I guess an alternative is to allow impl Trait in struct field types and elsewhere, so that in most places where you’d currently want to name a fn’s return type, you could just write impl Trait instead. In other words, impl Trait starts to act even more like its own “type”. Is that what you were thinking of? If so, that’s fine, and IMO better than encouraging use of typeof for this purpose. But I think it’s worth noting that that’s implicitly included in the vision.

[MajorBreakfast]

About "output of":

• I didn't notice it in the case of typeof because I see it everyday in JavaScript, but the two words concatenated together look a bit weird.
• I'd suggest to make it lowercase because it itself is not a type, it just returns a type
• It could be a compiler built-in: output_of!()

My preferred syntax is still fn_name<..>::Output where Output refers to the associated type of the FnOnce trait. It doesn't get any shorter.

@comex

One issue is that with a true typeof , you’d need turbofish: <typeof foo::<T>>::Output

You said that a while back. I've also never questioned why turbofish has the extra :: in there. Is it to avoid parsing ambiguities? Why do you think that that the turbofish operator should be used there?

[rpjohnst]

@MajorBreakfast

My preferred syntax is still fn_name<..>::Output where Output refers to the associated type of the FnOnce trait. It doesn’t get any shorter.

This would be committing to "a function's name is also the name of its type," which seems okay to me but also kind of a big deal? For one, that's currently not allowed- you need the <T as Trait>::Output syntax, so enabling that would be new. Second, error messages currently print out a function's type as fn(A, B) -> C {foo}, which seems important because it's close enough to the function pointer type syntax that you could guess that the coercion exists.

(Also, there's something about the current grammar not allowing turbofish in the middle of a path, but I'm not totally clear on when that applies.)

[comex]

@MajorBreakfast

You said that a while back. I’ve also never questioned why turbofish has the extra :: in there. Is it to avoid parsing ambiguities? Why do you think that that the turbofish operator should be used there?

Yes, it's to avoid parsing ambiguities between <> used for generics and as the less-than/greater-than operators. It shows up wherever generics are used in expression context.

[comex]

@rpjohnst

On top of what @withoutboats describes, typeof tends to accumulate a lot of hacks. The core problem is you have to be able to come up with a value for the type you want to talk about, which is not always easy or possible, which then leads to things like C++'s declval .

Fair point, though... A big part of the need for declval in C++ was the lack of another way to ask “what type does this function return, given these argument types?”. There’s now std::result_of for that, but its implementation just uses declval. In Rust, on the other hand, you can theoretically just project to a Fn* trait's Output associated type.

I say theoretically, because currently there's a pretty silly obstacle to doing so. You can write

<MyFuncTy as FnOnce<(u32,)>>::Output

but that depends on unstable (and ugly) syntax. You can't write

<MyFuncTy as FnOnce(u32)>::Output

because writing FnOnce(u32) implies a return type of (). Nor does using an underscore work:

<MyFuncTy as FnOnce(u32) -> _>::Output

I wonder if that could be made to work backwards-compatibly. That is, when the return type is _, instead of desugaring to FnOnce<(Args,), Output=_>, just desugar to FnOnce<(Args,)>.

@MajorBreakfast

The 'input lifetime is a brilliant idea! I think it’s even possible to add this in a 100% non-breaking way to Rust 2015. It can be defined by default and an explicitly defined lifetime called “input” simply shadows it (plus there should be a warning).

How about 'in? That's already reserved because in is a keyword, and it's more succinct.

[MajorBreakfast]

@comex

How about 'in ?

Attractive because it's so short. I like it.

'params would also work nicely. However, the shorter the better, I suppose.

[MajorBreakfast]

Here’s a code example: I hope I didn’t get anything wrong. I was obviously only able to check the currrently working stuff

Common: Setup code shared between all following code examples

struct MyStruct<T> { x: T }
trait MyTrait<T> { type Item; fn foo(&self) -> &T; }
impl<T> MyTrait<T> for MyStruct<T> { type Item = Box<T>; fn foo(&self) -> &T { &self.x } }

let my_struct = MyStruct { x: 42 };

Expression context: Referring to method without <.. as Trait>

let _ = <MyStruct<i32> as MyTrait<i32>>::foo(&my_struct); // Current
let _ = MyStruct<i32>::foo(&my_struct); // Proposed
let _ = my_struct.foo(); // Current: This already works for method calls

Type context: Referring to associated type without <.. as Trait>

let _ : <MyStruct<i32> as MyTrait<i32>>::Item = Box::new(123); // Current
let _ : MyStruct<i32>::Item = Box::new(123); // Proposed

Bad future

• We need <.. as Trait> to refer to a method or an associated type
• We need <typeof foo> to refer to the type of a function

let _ : <<typeof <MyStruct<i32> as MyTrait<i32>>::foo> as FnOnce(&MyStruct<i32>) -> &i32>::Output = my_struct.foo();

Looks like C++ template programming ^^’

(See the scrollbar? Did you use it? )

output_of

let _ : output_of!(<MyStruct<i32> as MyTrait<i32>>::foo) = my_struct.foo();
let _ : output_of!(MyStruct<i32>::foo) = my_struct.foo(); // Without `<.. as Trait>`

Question: I wonder: What’s actually inside of output_of?

• It’s not actually an expression because it’s incomplete
• Type also makes no sense because typeof and outputof are introduced to avoid making functions types

Edit: Answer by @withoutboats

Good future

let _ : MyStruct<i32>::foo::Output = my_struct.foo();

We can read from left to right, yay!

[MajorBreakfast]

@comex

Yes, it’s to avoid parsing ambiguities between <> used for generics and as the less-than/greater-than operators. It shows up wherever generics are used in expression context.

Perfect. Then we should be fine in <typeof ..> or with Type::Output without turbofish because those are type contexts. Edit: I can't decide what context is in inside <typeof ..> (I even said so in my previous post ^^')

BTW I see that nested comparisons without parenthesis lead to an error. Maybe turbofish can be removed in the future without making the parser type aware? Who knows... Doesn't matter for this discussion, though.