Pre-Pre-RFC: async methods & bounding async fns

language design
1

Motivation

The async fn RFC (in the process of being implemented) has two limitations:

  1. It only supports free async functions, not async trait methods (which allows async functions to play a role in our polymorphism system).
  2. It does not provide a way to require that your async fn’s future implements a trait like Send. This can often be very important.

These two problems were sort of related. This pre-RFC is a sketch of how they could be solved.

The main insight is that we can take inspiration from tuple structs, which create both a type and a function, and do the same thing for async functions.

Design

async fn creates a function and an abstract type

Today, async fn desugars to a function that returns an impl Trait. This means that async fn foo creates only an item in the expression namespace, not an item in the type namespace. Most kinds of item definitions in Rust only create a new item in 1 name space, but there is an exception: tuple structs, which create both a type and an expression - the expression is the constructor function for the type.

We could make async fn work similarly to tuple struct: create both an expression & and a type, the expression being the constructor for the type. In this case, the type is the future returned by async fn. This type would be an abstract/existential type. So instead of desugaring like this:

async fn foo() -> i32
/* desugars to: */
fn foo() -> impl Future<Output = i32>

We desugar it like this:

async fn foo() -> i32
/* desugars to: */
abstract type foo: Future<Output = i32>;
fn foo() -> foo

This allows the name foo to be used in type contexts to refer to the future that is constructed by the foo async fn. For example:

where foo: Send // bound that the future returned by foo implements Send

Async methods

Similarly, async methods desugar to an associated type and a method:

trait Trait {
     async fn foo(&self) -> i32;
}
/* desugars to: */
trait Trait {
     type foo<'a>: Future<Output = i32>;
     fn foo(&self) -> Self::foo<'_>;
}

There’s an important thing to notice here: the returned future captures all input lifetimes, therefore it needs to be a generic associated type. So async methods cannot be implemented until the compiler can comprehend them. (chalk can now, so its a matter of importing the concepts from chalk into the rustc type system.)

This allows downstream uses to require that the future returned by an async function implement a trait:

fn bar<T: Trait>(arg: &T) where T::foo: Send

Sugar for bounding the return type

While this system works great for adding that bound in a where clause, it is kind of awkward if you want to apply that check in the trait or function definition. Since no longer being Send would be a breaking change, users are likely going to want to automate the check that their API is Send if it is. That looks rather gnarly:

trait Trait where
     Self::foo: Send // note: must be on the trait, not method
{
     async fn foo(&self) -> i32;
}

// weird double reference to `foo` in this header:
async fn foo() -> i32 where foo: Unpin {
   // ...
}

As sugar for this, we allow the async keyword in a function header to take a parenthesized bound:

trait Trait {
     async(Send) fn foo(&self) -> i32;
}

async(Unpin) fn foo() -> i32 {
    // ...
}

If the trait contains the bound, all implementations must contain at least that bound (they can contain extra traits as well).

This bound is added to the bound on the existential type so that the function is checked that its future implements that bound.

Note: non-autotraits

At least at first, this would only be useful for Send and Sync, because they’re the only traits that will leak properly through the Future that the async fn evaluates to. However, this could be extended in the future:

  1. Similar to closures, we could make the future implement Copy and Clone if its entirely captured environment does.
  2. We could interpret Unpin specially to typecheck the Future as requiring that it be safe to move (futures without borrows).

However, neither of these are a part of this pre-RFC.

Alternatives

The main alternative is to only allow async fn as sugar in implementations, while requiring trait definitions to provide the whole associated type themselves. This seems very confusing and difficult to learn for users:

trait Trait {
     type Future<'a>: Future<Output = i32>;
     fn foo(&self) -> Self::Future<'_>;

impl Trait for Type {
     // declare the associated type abstract so it will be inferred:
     abstract type Future<'a>;

     // because this has the right signature, it is allowed even though
     // it is syntactically different from the trait definition:
     async fn foo(&self) -> i32 { /* ... */ }
}

This doesn’t solve the problem of bounding the return type of free async fns, which would still just desugar to impl Future.

7 Likes
2

This desugaring would also be useful for other reasons- storing the future in structs, implementing standalone methods and other traits for the generated type (potentially including #[derive]?), etc.

The async(Send) sugar seems like something that also might be useful in other scenarios. We seem to be doing okay with inferring Send/Sync for struct types and letting people track backwards compatibility manually, but it might be nice to pick a syntax here that applies to arbitrary type definitions. An attribute like #[derive] maybe?

3

Just as a note, it’d be great if whatever mechanism is used for async fn works for all return position impl Trait.

That is, I think async fn foo(&str) -> Bar should desugar to fn foo(&'a str) -> impl (Async<Output=Bar> + 'a) still (or whatever the decision was I didn’t keep track), which would then desugar to

abstract type foo<'a>: Async<Output=Bar> + 'a;
fn foo(&str) -> foo<'_>

Then you could simply desugar async(Unpin) fn foo() -> Bar as fn foo() -> impl (Async<Output=Bar> + Unpin) and async(Send) fn foo() -> Bar as fn foo() -> impl (Async<Output=Bar> + Send).

I have no real opinion on whether any of these traits should be inferred to be provided.

2 Likes
4

One issue with generalizing this sugar to any return-position impl Trait is that you can have multiple arbitrarily-nested impl Traits in a single function:

fn f() -> (impl TraitA, SomeType<impl TraitB>) { .. }

Which is why we need the abstract type feature to begin with- providing an independent name is the only way to make it work in general.

The single-impl Trait case could desugar to an abstract type as a special-case shortcut, but that’s not really necessary the way it is for async fn—you can’t write the abstract type version of an async fn, because it intentionally elides the impl Future<Output = > part of its type.

2 Likes
5

Ah, I see what you mean. I haven’t followed the abstract type discussion closely so missed that.

So existential impl Trait has to exist for when it’s nested. Still, it’d be very nice to be able to refer to the type of a returned impl Trait in this way, so there doesn’t have to be a dichotomy between functions declared as returning impl Trait or an explicit abstract type.

(Actually, is abstract type X: Trait<impl TraitB> valid syntax under abstract types?) (Not to derail further towards just impl Trait, just to acknowledge the point.)

1 Like
6

By extension from impl-trait-in-arguments I believe this should be allowed, and would desugar like

abstract type X<_0>: Trait<_0> where _0: TraitB;

except, that would block passing in the type for _0 and be unnameable in locations where it can’t be inferred.

7

Well, in case it wasn’t cited already, I see another alternative which would look like:

trait Trait {
    fn foo(&self) -> impl Future<Output = i32> + Send;
}

impl Trait for Type {
    async foo(&self) -> i32 { ... }
}

The trait def would desugar to:

trait Trait {
    type AnonymousType<‘a>: Future<Output = i32> + Send;

    fn foo(&self) -> Self::AnonymousType<‘_>;
}

and the impl to:

impl Trait for Type {
    abstract type AnonymousType<‘a>;

    fn foo(&self) -> /* put the (possibly compiler generated) right type here */ { ... }
}

It doesn’t seem confusing to me, the only mental overhead seems to be knowing that Future and async are somehow related, with async meaning « just let the compiler write Future things for me inside an fn body ».

8

That’s where I started too. There are two main problems with that approach:

First, your signature for the impl Future form is incorrect, because async fn & impl Trait have different lifetime elision defaults. The practical upside of this is that lifetime elision goes out the window to make these signatures match:

trait Trait {
    fn foo<'a>(&'a self) -> impl Future<Output = i32> + Send + 'a;
}

This is a big ergonomic and learnability problem with trying to make impl Future and async fn interchangeable: you have to understand the lifetime elision rules pretty intimately to know if you can actually interchange an fn -> impl Future with an async fn.

Second, because the associated type is anonymous, your trait provides no way for a downstream user to specify T where the return type of its foo impl implements another trait (like Sync or Unpin or Clone). Without some additional mechanism, using impl Future in the trait definition is strictly worse for downstream users than having an explicit associated type:

trait Trait {
    type Future<'a>: Future<Output = i32> + Send + 'a;
    fn foo(&self) -> Self::Future<'_>;
}

The culmination of both of these is that your alternative sort of devolves into the alternative I demonstrated: the trait definition has to contain an explicit generic associated type, use explicit lifetimes everywhere, and then only if they match correctly can an implementor replace all of that with async fn.

1 Like
9

Right, the lifetime elision thing seems confusing enough.

As for bounding the return type of an fn in an impl, do you mean that if a downstream user writes:

async(Clone) fn foo(&self) -> i32 { ... }

then only in that case the compiler would generate a type which implement Clone? You’d have to explicitly write that Clone instead of having the compiler always try to make that return type implement Clone if it can?

EDIT: oh ok, I see that when you have something simpler like:

fn foo() -> impl Fn(i32) -> i32 {
     |x| x + 1
}

Then you can’t do:

fn main() {
    let a = foo();
    let b = a.clone();
}

but I feel like you somehow should be able to do that…

10

I mean that a downstream user might want to write where T: Trait, T::foo: Clone, just as they might want to write where T: Iterator, T::Item: Clone. If the assoc type is anonymous, they can’t.

(They could if we had typeof, using a convoluted bound like <typeof T::foo as Fn>::Output: Clone).

11

Ok, now I understand why you’d like a named type. Thanks.

However in that case I would indeed find it weird that it only works for async methods and not for every method returning an impl trait, or at least a smaller subset of such functions, as @CAD97 was suggesting.

12

Does this work with generic functions as well? Like this?

async fn foo<T>(x: T) -> T
/* desugars to: */
abstract type foo<T>: Future<Output = T>;
fn foo<T>() -> foo<T>

I think that giving the function and its return type the same identifier is confusing. I’d really like if foo<T>::Output was a thing for all functions where the output type can be fully defined by the param types. (Note, that I didn’t follow the typeof debate so far. This suggestion might be bogus ^^’)

2 Likes
13

Currently it is not possible to use a function name as a type which would allow things like

struct Foo<F: FnMut(i32) -> i32> {
    //...
}

fn bar(x: i32) -> i32 {
    //...
}

impl Foo {
    fn new() -> Foo<bar> {
        //...
    }
}

This means we have to use impl FnMut(i32) -> i32 instead. This proposal would mean that this could no longer be implemented later. Maybe we could think about

async fn foo() -> i32
/* desugars to */
abstract type Foo: Future<Output = i32>;
fn foo() -> Foo

This would also be more consistent with other structs that use PascalCase instead of snake_case. However it may be to much magic for the compiler to switch to PascalCase automatically.

Edit: We could also use foo::Output as @MajorBreakfast suggested.

2 Likes
14

Yep. I've experience with automatic name conversions from web development. Let's not open Pandora's box xD

5 Likes
15

Yes I like your foo::Output more, we already have

pub trait FnOnce<Args> {
    type Output;
    extern "rust-call" fn call_once(self, args: Args) -> Self::Output;
}

As the definition for FnOnce.

1 Like
16

However, while accessing an associated type works fine for structs (playground link), I can’t access it on a closure (playground link) or a function (playground link) because they aren’t types. And they probably shouldn’t be types, but it’d be great if there was a convenient way to get the associated type. (Edit: I just mean that that deserves its own discussion)

Edit: Isn’t there a way to make the (currently not possible) <typeof T::foo as Fn>::Output that @withoutboats mentions above nice? Just T::foo::Output would be very nice.

17

As far as I can tell there are 2 open issues: First, given

trait Trait {
    async fn foo() -> i32;
}

foo must be implemented in exactly the specified way, i.e., can’t choose to implement foo using a normal fn that returns an async block (?).

Second, isn’t the same discussion bound to happen for generators as well?

For these reasons I still have reservations about the fact that async conflates 2 things: conversion of the fn body into a closure, and adapting the return value, including lifetimes. Hence I’m wondering if we shouldn’t separate those 2 concerns, for example:

fn foo<T>|&self| -> (async T + Send) + Send {
    //   ^           ^
    //   |           +-- conversion of the result type, and lifetime handling
    //   +-- conversion of the function body into a closure
    ...
}

With this, the example trait would look like

trait Trait {
    fn foo(&self) -> async i32;    // async only in trailing position (!)
}

This can be implemented both ways:

// ```rust doesn't handle syntax highlighting well here
impl Trait for TypeAsyncFn {
    fn foo|&self| -> async i32 { self.x };
}

impl Trait for TypeAsyncBlock {
    fn foo(&self) -> async i32 { 
        // ... do some initialization
        async { self.x } 
    }
}
18

The RFC defines async functions as completely ordinary functions which return an anonymous type that implements Future.

The notation that @withoutboats mentions is just a shorthand that was discussed in the RFC thread. It is not part of the RFC, but having it is probably worth it. This comment by @Nemo157 explores why. The two notations are exchangeable if you get the lifetimes right. (Unless there has been some discussion about it outside of the RFC thread that I'm not aware of)

Edit: async i32 in your code should be Future<Output = i32>. Currently, Future is going to be added to std as Future See the most recent proposal by @cramertj. (probably without the Unpin bound mentioned in the comment, though) Please note, that the details currently change almost every week ^^' At the moment there are even two branches for two different approaches: "0.3" and "0.3-PinFuture" (the latter is for the proposal by @cramertj I just mentioned)

19

Yes, but unless I'm completely misunderstanding something, creating that anonymous state machine closure type is also done via the very same async annotation. Conversely, when I want to return an async block from a function I cannot use the same async sugar (unless something changed since the original thread) (?).

Actually, shouldn't it be Future<Output=i32> + 'a , where 'a is from & 'a self? That's actually one of the points I wanted to make: to make the async sugar a bit more universally usable. Shortened:

fn foo() -> async i32    the function body executes immediately and returns a Future
                         (likely by returning an async block), lifetimes are handled

async fn foo() -> i32    the function body is itself transformed into a closure that is
                         initialized and returned, but no other code runs initially.

However, with this, the first version would then make more sense to use in a trait declaration.

20

You need to differentiate between function declaration and definition. The declaration (just signature) can be written with or without the sugar. The definition is what compiles to an ordinary function that returns the state machine. Every async fn can also be defined as a normal fn (tricky lifetimes!) that returns an async block. Although, actually doing that makes only sense when you want to execute part of it synchronously. (initialization pattern)

Yes, and impl in front of it is also required. See the comment by @Nemo157 for examples that contain lifetimes. All I meant is that async T does not exist in the signature in the return position.

1 Like