[lang-team-minutes] Elision 2.0

[leodasvacas]

Has it been discussed to simply elide the lifetime completely?

Could be plausible if you always know what the fields are, but when using a struct from another crate the docs will say: pub struct Foo { /* fields omitted */ } and now you have no idea that the struct has a lifetime parameter. So we either leak private details or introduce new notation, which is what the annonymous lifetimes proposal does.

As a new user, it seems obvious that Bar values will have a lifetime associated with them – expect all values to have a lifetime.

All values have lifetimes which is the scope where they are accessible, but struct Foo is a type not a value, and struct Foo<'a> is a type generic over a lifetime parameter. Not all types have lifetime parameters, a value of type Foo<'a> cannot outlive the 'a lifetime while a value of type Foo has no restriction on it’s lifetime.

[withoutboats]

As a new user, it seems obvious that Bar values will have a lifetime associated with them – expect all values to have a lifetime. Since Bar contains a Foo, I would expect the lifetime to be determined by Foo.

To give a concrete example of why your intuition is here is wrong, given this API:

struct Foo(i32)

struct Bar(&i32);
struct Baz(i32);

impl Foo {
     fn bar(&self) -> Bar { Bar(&self.0) }
     fn baz(&self) -> Baz { Baz(self.0) }
}

// Invalid.
// This would be moving a reference into "Foo(0)" out of this function.
fn make_bar() -> Bar {
    Foo(0).bar()
}

// Valid.
// All of the data moved out of this function is owned.
fn make_baz() -> Baz {
    Foo(0).baz()
}

Nothing in the API distinguishes Bar and Baz from one another, but they are very different from one another. What this parameter distinguishes, in high level terms, is whether or the type owns all of the data, or if it is only borrowing some of it. If it has a lifetime parameter, it is bound to the lifetime of another value which it is only borrowing, so its lifetime has to be shorter than that value’s.

This is why we feel its very important to document this fact in the API.

[mgeisler]

This is why we feel its very important to document this fact in the API.

Wow, thank you very much for that example! This community is great – always patient and with great explanations. It makes me feel very welcome

Just for completeness, the API you described above was in my proposed syntax. In real Rust, I believe it would look like this:

struct Foo(i32);

struct Bar<'a>(&'a i32);
struct Baz(i32);

impl Foo {
     fn bar<'a>(&'a self) -> Bar<'a> { Bar(&self.0) }
     fn baz(&self) -> Baz { Baz(self.0) }
}

// Invalid.
// This would be moving a reference into "Foo(0)" out of this function.
fn make_bar<'a>() -> Bar<'a> {
    Foo(0).bar()
}

// Valid.
// All of the data moved out of this function is owned.
fn make_baz() -> Baz {
    Foo(0).baz()
}

The make_bar function is now more clearly invalid since there’s nothing with a lifetime of 'a going into it. It cannot simply conjure up and return a value of limited lifetime. (Why not? One reason, I believe, is that it wouldn’t be clear who owns the borrowed value inside the Baz value. Without an owner, there’s nobody to free the borrowed memory.)

It would instead have to be written like this (if the Bar is to be derived from a Foo):

fn make_bar<'a>(foo: &'a Foo) -> Bar<'a> {
    foo.bar()
}

[withoutboats]

One reason, I believe, is that it wouldn’t be clear who owns the borrowed value inside the Baz value. Without an owner, there’s nobody to free the borrowed memory.)

This is basically correct with one amendment: the borrowed value is owned by the Foo(0) constructed inside make_bar, which is freed when the function concludes (so the borrow is outliving the owned value). If you know C, this is the same error as returning a pointer to a value allocated on the stack during that function call.

[leodasvacas]

Thanks for bringing this up! I wanted to mention that as a possibility. That said, I don’t think I’ve seen a syntax I prefer yet. My expectation is that actually giving explicit names to lifetime parameters will become quite rare, so I would want the syntax to be very lightweight; this seems to rule out (for example) ''x. '(x) is a possibility, I suppose, but it still feels like it biases the wrong way.

I like this a lot! What about a'?

[Fylwind]

If the elided form ends up being as or more verbose than the unelided form, one may as well just write out everything using the regular unelided syntax because the latter has the benefit of being consistent and more flexible.

[leodasvacas]

I realized we are thinking of different things. I agree that parameters as lifetimes will end up being too verbose. I’m thinking of introducing syntax for undeclared lifetimes that work just like explicit lifetimes but you don’t declare them, and they can appear only in an fn signature, as in: fn foo(&self, data: &a' [i32]) -> Foo<a'> which is a small win over: fn foo<'a>(&self, data: &'a [i32]) -> Foo<'a>

[burakumin]

An example should explain. Consider this snippet:

impl Bar {
    // Here, the result references the argument `data`, so we tag them both with `'a`.
    fn foo<'a>(&self, data: &'a [i32]) -> Foo<'a> { ... }

    // Here is an alternative, using the new feature:
    fn foo(&self, data: &'data [i32]) -> Foo<'data> { ... }
}

Am I guessing correctly that the second example should just be fn foo(&self, data: &[i32]) -> Foo<'data> { ... } without the first 'data?

[leodasvacas]

Rereading the original proposal, I noticed I was getting it wrong. I thought that an annonymous lifetime would match another one, and the following would be valid:

fn foo(x: &Bar, y: Foo<'>) -> Foo<'>

But that is not the case, an annonymous lifetime is merely a marker for an elided lifetime, it can only make the syntax longer and never shorter. I had a different interpretation (which I find more useful) that an annonymous lifetime was just like any other except that it had a special syntax and didn’t have to be declared.

We could reinterpret the annonymous lifetime syntax as something I’ll call undeclared lifetimes. They would be just like explicit lifetimes but they are not declared and may appear only in fn signatures. They must be unambiguous with explicit lifetimes, for example '0, '1 could be the syntax for undeclared lifetimes.

This is how the above example would look:

// Leaving &Bar elided.
fn foo(x: &Bar, y: Foo<'0>) -> Foo<'0>
// Giving &Bar a throwaway lifetime.
fn foo(x: &'1 Bar, y: Foo<'0>) -> Foo<'0>

In comparison to parameter as lifetime:

// Parameter as lifetime.
fn foo(&self, data: &[i32]) -> Foo<'data>
// Throwaway lifetimes.
fn foo(&self, data: &'0 [i32]) -> Foo<'0>
// Parameter as lifetime cannot handle this case.
fn foo(&self, data: &[&'0 i32]) -> Foo<'0>

But most of the time we are only using '0, can we get sugar for that? Here we may repurpose the annonymous lifetime syntax to be sugar for an undeclared lifetime, possibly only when it is the only unelided lifetime in the signature, now the original example is valid:

// These could work.
fn foo(x: &Bar, y: Foo<'>) -> Foo<'>

fn foo(&self, data: &' [i32]) -> Foo<'>

impl<'a> Bar<'a> {
    fn foo(&self, data: &'0 [i32]) -> Foo<'0, 'a>
}

// This style got more verbose.
fn foo(x: Foo<'0>, y: Foo<'1>)

// We probably don't want to mix things too much.
fn foo(&'0 self, data: &' [i32]) -> Foo<'0, '>

What do you think?

[nikomatsakis]

Am I guessing correctly that the second example should just be fn foo(&self, data: &[i32]) -> Foo<'data> { … } without the first 'data?

Yes.

[nikomatsakis]

So I just had a conversation with @wycats. We elaborated on “the syntax table”, and we also had the insight that – as a general rule – if you have an &Foo, we can still completely allow eliding any lifetime parameters of Foo, since there is already an indication of borrowing from the outer & (this was the same argument that I used to claim we would not need to ever write &Foo&).

// Not eliding 'nested lifetimes' like `'a` in `&Foo<'a>`
Foo'     Foo'<T>     &Foo'        &Foo'<T>
Foo&     Foo&<T>     &Foo&         &Foo<T>
Foo<'>   Foo<', T>   &Foo<'>      &Foo<', T>
Foo<&>   Foo<&, T>   &Foo<&>      &Foo<&, T>
Foo<ref> Foo<ref, T> &Foo<ref, T> &Foo<ref, T> 

// Eliding 'nested lifetimes' like `'a` in `&Foo<'a>`
Foo'     Foo'<T>     &Foo         &Foo<T>
Foo&     Foo&<T>     &Foo         &Foo<T>
Foo<'>   Foo<', T>   &Foo         &Foo<T>
Foo<&>   Foo<&, T>   &Foo         &Foo<T>    // my preference
Foo<ref> Foo<ref, T> &Foo<T>      &Foo<T> 

When I look at these syntaxes, particularly the second set, I feel like Foo<&> stands out to me as the best choice:

• we have the consistency of & as the “elidable lifetime”
  • when first learning, you can probably get quite far this way
• using the angle brackets, like Foo<&>, strongly reads to me as “struct with references in it”, which is precisely what it is
  • that is, we do not have the &Foo vs Foo& confusion
• we do not have the “floating single tick” syntax (') that is very surprising and provokes strong reaction in some people
  • we also avoid “doubling down on the tick”

The only case in that table that kind of “looks heavy” to me Foo<&, T> – but it’s also one of the cases where I personally get confused about today (because you can elide the &). (Well, I sometimes get confused about &Foo<T>, but I think the ongoing work on improved lifetime errors will solve this.) It feels like this comes up rarely to me in practice; it’d be interesting to try and measure it.

I agree that there is a nice continuity between Foo<', T> and Foo<'a, T>, but it feels like it is compensated for by the twin downsides of:

• single tick alone is very jarring to some people
• lack of continuity between &T and Foo<&>

[petrochenkov]

if you have an &Foo, we can still completely allow eliding any lifetime parameters of Foo, since there is already an indication of borrowing from the outer &

Could you elaborate, what is the logic here? I just… don’t see any connection. What difference does the outer & make? If Foo<'a> is “good” and Foo is “bad”, then why is &Foo good again? What is the purpose of &/' if not “highlighting” types containing lifetimes?

[nikomatsakis]

Could you elaborate, what is the logic here? I just… don’t see any connection.

The way I see, if you have &Foo<&>, but you do not write the “inner” & (i.e., you write &Foo), then you already have a signal that the Foo is borrowed, and you would not expect to be able to (e.g.) take ownership of it, at least not without knowing more details about the type. Moreover, if you see it in return position, then you would already know that the lifetime of self is extended to cover the use of the return value.

In short, I want to ensure that there is a & or 'x somewhere when there are borrowed values floating about, but I’m not sure they have to be everywhere. The case that I think is actively confusing is fn foo(x: Ref<T>) or fn foo(&self) -> Ref<T>, where there is no sign at all that the argument contains borrowed data. If you see fn foo(x: &Ref<T>) or fn foo(&self) -> &Ref<T>, I think that is much better. (But feel free to tell me you disagree.)

That said, I am not sure if same logic applies when there are named lifetimes involved. For example, if you have these signatures, there is more room for surprise:

impl<'a> Foo<'a> {
    fn foo(&self) -> &'a Ref<T> { ... }
    // becomes: fn foo<'b>(&'b self) -> &'a Ref<'b, T> { ... }
    fn bar(&self, x: &'a Ref<T>) { ... }
    // becomes: fn bar<'b>(&self, x: &'a Ref<'b, T>) { ... }
}

This perhaps falls under the general header against mixing named and elided lifetimes, although neither is quite the cases I had in mind (since the named lifetime is in a different “layer” of the type).

[leodasvacas]

The case that I think is actively confusing is fn foo(x: Ref<T>)

The input lifetime case could be solved with a syntax highlighter that colors structs with references differently, or underlines them or something. Sufficently smart IDE is usually a bad argument, but in this case it’s a simple IDE feature that avoids a breaking change to syntax.

EDIT: My wording is incorrect, as Niko points out below this is not a syntax breaking change only a syntax deprecation, but the point still stands.

[KasMA1990]

So just to be clear, your preferred solution would also excise the tick from named lifetimes (so Foo<'a, T> would become Foo<&a, T> if you want the lifetime named), right?

[nikomatsakis]

Sufficently smart IDE is usually a bad argument, but in this case it’s a simple IDE feature that avoids a breaking change to syntax.

To be clear: nobody is proposing a breaking change. All existing code would continue to work, though some patterns might be deprecated (and perhaps automatically transitioned by a rustfix-like tool, but that’s a separate thread). I am not sure how “active” this deprecation has to be-- i.e., should we actively warn, or just teach it another way?

So just to be clear, your preferred solution would also excise the tick from named lifetimes (so Foo<'a, T> would become Foo<&a, T> if you want the lifetime named), right?

No, if you use a named lifetime, you would use 'a just as today. The & would just be for the case where you elide things. And this is the one downside, I think: that for &T, you “add” the 'a to get &'a T, but for Foo<&>, you “replace” the 'a with Foo<'a>. Writing Foo<&'a> would be ambiguous in any case.

To be clear, I see the appeal of the ', and there is something nice to “just elide the name”, but I don’t it’s an open-and-shut case as to which is better. They each have their advantages, as @glaebhoerl pointed out earlier. (Among other things, using ' also has a kind of discontinuity, since you don’t write &' T when you elide the lifetime from &T, but you do write Foo<', T>.)

[phaylon]

To me, Foo<&> is more implying “parameterized by a reference”, but I already said as much way up there.

Some other downsides:

• Currently, I see & in a signature and I expect that to be a type.
• I could see learnability problems wrt. & vs &mut if the former is repurposed.

[RalfJung]

The way I see, if you have &Foo<&>, but you do not write the “inner” & (i.e., you write &Foo), then you already have a signal that the Foo is borrowed, and you would not expect to be able to (e.g.) take ownership of it, at least not without knowing more details about the type. Moreover, if you see it in return position, then you would already know that the lifetime of self is extended to cover the use of the return value.

On the other hand, this makes the rule for having to write <&> context-dependent. That’s going to cause confusing on its own, and I am not sure that’s worth it here. I feel things are more consistent when adding or removing an & doesn’t also magically change which annotations you have to (not) make “below” this type operator.

That said, I find myself somewhat disagreeing with the entire premise of the <&> proposal. I was actually positively amazed when I realized that lifetime elision “just works” in fn foo(&self) -> Ref<T>. I never considered this confusing. After all, it says Ref right there, which is a pretty good indication of a reference, isn’t it? Of course, API designers could make this confusing by picking bad names, but that is not something a compiler can fix. So, I would be somewhat sad if this got deprecated. But maybe I just didn’t run into this issue yet because I didn’t work with enough code that puts lifetimes in structs in interesting ways.

[krdln]

Currently, I see & in a signature and I expect that to be a type.

That made me think – is the Foo<&> syntax forward compatible with higher kinded types? Imagine that if Foo had a kind: lifetime -> type -> type, then the Foo<&> syntax could be valid usage of & type constructor. I’m in favour of the Foo<&> meaning elided lifetimes, but I just want to make sure it won’t creates syntatic ambiguities with other proposed features.

[nikomatsakis]

On the other hand, this makes the rule for having to write <&> context-dependent. That’s going to cause confusing on its own, and I am not sure that’s worth it here.

That’s fair. I don’t know for sure if it is or not. I think what I would probably prefer is that I can type whatever I want, but rustfmt fixes it up for me to the canonical form (right now we often do this sort of thing with lints that “steer” you the right way, and this might be a good place for that too, at least until we get a smarter rustfmt).

After all, it says Ref right there, which is a pretty good indication of a reference, isn’t it?

You can memorize Ref, but there are many other types with lifetimes. What about MutexGuard? Imagine you don’t really know how mutex works… in that case, there is nothing in this signature that would tell you that invoking lock() is going to make the mutex be considered borrowed for the lifetime of the return value:

fn lock(&self) -> MutexGuard<T>

Add to that things like vec::Iter (borrows) but vec::IntoIter (does not).

Then you get into projects like the compiler, which make extensive use of references. When you see a signature like this:

fn cause(&mut self, code: traits::ObligationCauseCode) { ... }

doesn’t it seem a bit surprising that you can’t (e.g.) return this code out from the function, or make a struct and store it? In contrast:

fn cause(&mut self, code: traits::ObligationCauseCode<&>) { ... }

makes it pretty clear that you won’t be able to do that (in that particular case, I admit, it’s most common in the compiler to see ObligationCauseCode<'tcx>, but it’s still fairly common for me to find signatures that elide any reference to the lifetime because it doesn’t happen to matter in that particular spot). This can be particularly confusing because the compiler undergoes regular refactorings, and things that used to have references sometimes change so that they no longer do, or vice versa.