[lang-team-minutes] stabilization and future of impl Trait

[jethrogb]

Can you write something like Vec<fn() -> impl Trait>? If so, what does it mean?

[nikomatsakis]

You can, but only in some contexts. For example:

fn foo() -> Vec<fn() -> impl Trait> { ... }

if we permitted explicit “out” parameters, this would be desugared to something like

fn foo<out T>() -> Vec<fn() -> T>
    where T: Trait

In other words, there is some type T, known to foo but not its callers; T implements Trait; and foo() returns a vector of functions that return T.

(Not that I would necessarily advocate that syntax.)

[Ixrec]

Maybe this is just me, but given how many unanswered questions there seem to be about the impl Trait syntax, and how few questions there seem to be about the semantics of anonymous return types themselves, I feel like it might be beneficial to treat “impl syntax” as a completely separate feature from anonymous return types.

fn foo() -> _: Trait strikes me as the most natural alternative syntax for anonymous return types, since _ (when used as a type) already means “compiler, please infer this type for me”. That would also allow us to specify details like lifetime parameters exactly the same way we currently do, without having to invent or teach a new set of type parameter inference rules. Based on the RFC thread, it seems like no one ever suggested using _, much less argued for or against it (the syntax debate that I did find was focused on impl Trait vs @Trait). Is there some technical reason I’m not aware of that requires us to use a brand new keyword for implementing anonymous return types, or is it just that anonymous return types seemed like a convenient way to start introducing “impl syntax”?

[arielb1]

@nikomatsakis

I don't quite follow. It seems to me that impl (Trait+'a) (using parens to clarify precedence) already has a meaning: it requires that the hidden type outlives 'a. (Note that this may in turn influence inference; clearing up the inference impl issue is actually another thing I should have brought up in the lifetime meeting.) However, it seems orthogonal to the problem at hand. In particular, it doesn't provide a way to talk about the type parameters that get captured (or not captured, as the case may be.)

But the only thing interesting in captured type parameters is their lifetime. You can add a lifetime 'ret and force all type parameters to outlive it.

[notriddle]

That syntax implies that you the human can fill in something for the _ while keeping the rest of the syntax the same, which isn’t true (unless I’m very wrong and you actually do want to allow fn foo() -> slice::Iter<usize>: Iterator<usize>). And it still doesn’t allow the sort of thing that fn foo() -> <T: Trait> T allows (which was already suggested, IIRC, because of that).

[nikomatsakis]

@arielb1

But the only thing interesting in captured type parameters is their lifetime. You can add a lifetime 'ret and force all type parameters to outlive it.

Well, another thing that might be affected is equality.

For example:

fn foo<T,U>(t: T, u: U) -> impl<T> Trait { ... }

fn bar<T>(t1: T, t2: T) {
    let mut x = foo(t1, 22);
    if some_condition() {
        x = foo(t2, 'a'); // this would be a type error if `foo()` returned `impl<T,U> Trait`
    }
}

[withoutboats]

Is it unfeasible to determine while typechecking a function that returns an impl trait which parameters are captured in its return value without the users providing any manual annotation? That seems very appealing to me.

[eddyb]

That sounds like global inference, which is what this anonymized types design is trying to avoid, it’s not just an abstraction tool, its opaque (modulo OIBITs) semantics are required for inference isolation and efficient implementations.

That said, I was hoping OIBITs could leak gracefully but they seem like a mess TBH.

We probably need to make them very strict and/or have DAG relationships - I still hope there’s a way to make the whole OIBIT thing “error or not, but don’t add inference information” even if the concrete type is necessary - which is hard, more so because of lifetimes than anything else.

[gnzlbg]

My 2 cents:

Before stabilizing the minimal impl trait in for free function return types we should have implemented in unstable without going through RFCs:

• a way to specify life-times for free function return types (using impl Trait),
• impl Trait in argument position, probably with life-time annotations as well, and
• impl Trait for trait method return types.

Furthermore,

• we should have achieved consensus about the default life-time elision rules of impl Trait in free function return types, and
• it would be nice if we have an implementation of impl Trait for local variables (e.g. something accepting the result of a function returning an impl Trait), but this does not need to be in unstable.

The only RFC that has gone through is “minimal impl trait” without explicit lifetime annotations, so that is actually the only thing that, in my opinion, is on the table for stabilization.

Still, since we want all these other extensions anyways, there is no real way of stabilizing it without at least having played with the extensions, which are many. At the same time, since the purpose of implementing the extensions is just to be sure that there isn’t anything fundamentally wrong with the proposed “minimal impl trait”, I do not think that we need RFCs for the extensions just yet, only implementations. These implementations will make it easier to write future RFCs for the extensions though, so that is a plus.

[kylone]

For the capturing syntax, if we’re feeling uncomfortable with impl<> Trait, we always could use impl<()> Trait.

[nikomatsakis]

@withoutboats

Is it unfeasible to determine while typechecking a function that returns an impl trait which parameters are captured in its return value without the users providing any manual annotation? That seems very appealing to me.

It is feasible, but is it desirable? @eddyb brings upsome impl specifics, but I am not so concerned about that. I think impl Trait as specified already crosses the bridge of global inference (due to auto trait leakage) and I'm ok w/ that (though it's a good point that we really have to nail that down in the impl before stabilizing, since it likely implies enforcing a kind of DAG through the graph).

However, there are other reasons to be against this sort of implicit inference. In particular, your clients are now relying on details of your implementation in subtle ways. Just because you don't happen to use data from a particular reference now in your return type doesn't mean you want to promise you never will in the future.

[withoutboats]

Got a question I want to get a clearer sense of: impl Trait in trait bodies.

tokio contains the Service trait, which currently is defined like this:

pub trait Service {
    type Request;
    type Response;
    type Error;
    type Future: Future<Item=Self::Response, Error=Self::Error>;
    fn call(&self, req: Self::Request) -> Self::Future;
}

Ultimately, high level application authors are probably going to be implementing some other trait, which specifies every associated type here aside from Future. However, each individual implementation is going to have a unique future; often, this future will be some sort of gnarly combinator, the details of which are unimportant to the user. Any time they add an and_then call in their stack, they’ll need to fiddle with the type of Self::Future.

This is exactly the problem impl Trait is supposed to solve. Not to be too pessimistic, I don’t see how any future based framework will be useable without access to impl Trait.

So what exactly are the blocking issues to impl Trait in trait bodies, and what can we do to resolve them?

[eddyb]

Do note that in order to use impl Trait in the trait impls, it’s not necessary to get rid of the associated types. AFAIK there’s no blockers there (and no questions of lifetime elision either), just needs to be proposed and accepted.

It was implemented last year in the first impl Trait prototype, and the implementation cost is negligible IMO.

[withoutboats]

@eddyb

Do note that in order to use impl Trait in the trait impls, it's not necessary to get rid of the associated types.

What do you mean? Something like assigning the associated type to an impl trait?

@eddyb

AFAIK there's no blockers there (and no questions of lifetime elision either), just needs to be proposed and accepted.

I thought it was functionally ATCs or something like that?

[eddyb]

I’m talking about type Future = impl Future<Item=T, Error=E>;. Doing this in the trait impl would allow you to return any Future from the method without losing access to the type through the associated type of the trait and without accidentally introducing ATC-like capabilities.

Writing impl Trait anywhere inside the trait is indeed related to ATC, and is a different feature (associated type sugar) than the one which was implemented.

In fact, writing impl Trait in both the trait and the impl would be a combination of these two features, an anonymous associated type for the trait’s declaration, fulfilled by an anonymized type in the impl’s definition of the same method.

[withoutboats]

Just as a motivating example, I am trying to convert a mildly complicated call from a Result to an associated IntoFuture type, to be compatible with tokio. This is not a very deep future stack, production applications will have much more complex combinators (you can see in the error output that the code being typed fits on 1 line). This is the kind of error I am getting because I wrote the annotation wrong:

   Compiling cargonauts v0.1.1                                                                    
error[E0308]: mismatched types
  --> src/api/rel/fetch.rs:24:9
   |   
24 |         <T as HasOne<Rel>>::has_one(entity).into_future().and_then(not_found).join(Ok(includes)).and_then(fetch_one)
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected fn pointer, found fn item
   |   
   = note: expected type `futures::AndThen<futures::Join<futures::AndThen<<<T as api::rel::HasOne<Rel>>::HasOneFut as futures::IntoFuture>::Future, std::result::Result<<<Rel as api::rel::Relation>::Resource as api::Resource>::Id, api::error::Error>, fn(std::option::Option<<<Rel as api::rel::Relation>::Resource as api::Resource>::Id>) -> std::result::Result<<<Rel as api::rel::Relation>::Resource as api::Resource>::Id, api::error::Error>>, futures::Done<std::vec::Vec<router::include::IncludeQuery>, api::error::Error>>, <<Rel as api::rel::Relation>::Resource as api::get::RawGet<I>>::IdFuture, fn((<<Rel as api::rel::Relation>::Resource as api::Resource>::Id, std::vec::Vec<router::include::IncludeQuery>)) -> <<Rel as api::rel::Relation>::Resource as api::get::RawGet<I>>::IdFuture>`
   = note:    found type `futures::AndThen<futures::Join<futures::AndThen<<<T as api::rel::HasOne<Rel>>::HasOneFut as futures::IntoFuture>::Future, std::result::Result<_, api::error::Error>, fn(std::option::Option<<<Rel as api::rel::Relation>::Resource as api::Resource>::Id>) -> std::result::Result<_, api::error::Error> {api::rel::fetch::not_found::<_>}>, futures::Done<std::vec::Vec<router::include::IncludeQuery>, api::error::Error>>, _, fn((_, std::vec::Vec<router::include::IncludeQuery>)) -> _ {api::rel::fetch::fetch_one::<_, _>}>`

error: aborting due to previous error

error: Could not compile `cargonauts`.

To learn more, run the command again with --verbose.

This is setting aside the fact that because I cannot express a closure’s anonymous type in the associated type (AFAIK), I have to define helper functions and pass environment variables into the function explicitly using a join combinator.

---

@eddyb Doesn’t allowing impl trait in associated type positions push it even closer to some global inference? That associated type could appear in multiple positions in the function body, both input and output.

[eddyb]

It never gets any closer unless it can bridge between inference contexts. That is, if multiple methods can independently determine the concrete type, without seeing eachother’s results, it’s not global inference.

[withoutboats]

In addition to the basic ergonomics issue of “defining this type is extremely unpleasant” I’ve encountered some other issues with having the future as an associated type:

1. Because closures have an anonymous type, you can not parameterize this associated type by a closure. Therefore, you have to use a named function as a function pointer. This not only introduces a virtual call through a function pointer, making the code less efficient, but also requires you to pass variables from the environment into the function using a join call, making your type even gnarlier.
2. You cannot use a borrowed type in the associated future, because the lifetime would be unbound. This may also make your code less performant, because you may need to perform deep clones to create a viable type.
3. You effectively cannot specialize a function returning an associated future type, because they would return different types.

The only option right now seems to be to use trait objects.

[arielb1]

@withoutboats

You cannot use a borrowed type in the associated future, because the lifetime would be unbound. This may also make your code less performant, because you may need to perform deep clones to create a viable type.

That would require ATCs to solve - type Future<'a> where Self: 'a;.

@withoutboats

You effectively cannot specialize a function returning an associated future type, because they would return different types.

That's... I am quite sure we have associated type specialization implemented. Can't you use it?

[withoutboats]

@arielb1

That would require ATCs to solve - type Future<'a> where Self: 'a;.

That's correct - this is why impl Trait in trait bodies is blocked on ATCs.

@arielb1

That's... I am quite sure we have associated type specialization implemented. Can't you use it?

I thought the RFC concluded with not supporting specializing associated types (because of the way that influences whether or not you need to specialize other associated items in the impl). I just messed around with it in playpen and while you can specialize associated types, it doesn't seem possible to actually use them: Rust Playground