Why does Rust require explicit lifetimes?

I've on-again and off-again picked up Rust tutorials for the past 7 or so years but haven't had an opportunity to use it for either my job or any major projects yet. In the 10 or so times I've picked up the language and built a silly little Fibonacci calculator or other tool for fun, I always get tripped up by the same issue. The language requires me to provide explicit lifetimes even when, as far as I can tell, they should be obvious!

I'll pull an example from The Rust Book:

fn longest(x: &str, y: &str) -> &str {
    if x.len() > y.len() {
        x
    } else {
        y
    }
}

In this function definition it's unclear whether the reference returned by the function will have the same lifetime as x or the same lifetime as y. In fact, you literally can't know -- it could be either!

But here's what we do know: the returned reference will live at least as long as the shorter-lived of the two! So even without declaring any lifetimes, I know the following:

fn main() {
    let string1 = String::from("abcd");
    let result;
    {
        let string2 = String::from("xyz");
        result = longest(string1, string2);
        
        println!("{}", result); // both string1 and string2 are still in scope, so this will ALWAYS work
    }
    println!("{}", result); // string2 is out of scope, so this MIGHT fail (unknown)
}

For single-input functions Rust already has a lifetime elision rule to handle this. But for multi-input functions, a decision was made to require explicit lifetimes. This feels like extraneous work for developers.

If you always assume the shorter lifetime of the two inputs is the lifetime of the result, then you can never run into a situation where the borrow checker fails to catch an error.

Doing lifetime elision like this also wouldn't preclude developers from choosing to override the default lifetimes when they know better. For instance:

fn one_char<'a>(a: &'a str, b: &str) -> &'a str {
    if b == "first" {
        return &a[0..0];
    } else {
        return &a[1..1];
    }
}

fn main() {
    let input = String::from("test"); // lifetime 'a
    let result;
    {
        let decider = String::from("first");
        result = one_char(input, decider);
    } // decider falls out of scope, but the borrow checker knows result has a lifetime of 'a
    println!("{}", result); // no errors
}

I've tried Googling but can't find a good justification for why "assume the shortest lifetime" elision like this isn't done. I also tried asking ChatGPT but it went in circles reiterating the "longest" example from the Rust Book but never actually explaining the reasoning.

You can not assume "out of N passed-in borrows, return shortest", it's obviously nonsense for most real usecases.

2 Likes

This is roughly equivalent (for covariant lifetime parameters) to saying “assign one elided lifetime to all input parameters and the return value”. If you do that, then people will often end up writing over-constrained functions, where lifetimes are forced to be equal/shorter when they don't have to be. This is not intrinsically worse than under-constraint, but the error messages are worse. When you get “does not live long enough”, due to elided lifetimes, the fix is often to make the two mentioned lifetime parameters equal — written as replacing elided lifetimes with non-elided ones. When lifetimes are already equal, you get much more “action at a distance” — you have to figure out which lifetime equality of the many involved is connecting two things that should be different.

Look at all the head-scratching associated with the “you tried to make a self-referential struct” problem — the weird error messages you get there are similar to you’d be having on everything here.

Perhaps the compiler could be doing better, in that world, but perhaps not.

7 Likes

Could you explain in a bit more detail?

For starters, I recommend only assuming "return shortest lifetime" as the default case, and allowing developers to override this assumption explicitly if they know that, for instance, the returned reference will live as long as the first argument (see my example with one_char)

Secondarily, I don't understand why it's "nonsense". Are you saying that there's no unambiguous way to know the shortest lifetime and the problem is therefore not representable in the borrow checker? If so, please provide a clear example of when lifetimes of the input parameters to a method call would not be known. Or are you saying that it's functionally nonsense because it doesn't cover most use cases? If so, please see my previous statement about retaining the lifetime syntax for developers to explicitly override the default behavior when my proposed elision rule is too restrictive.

I really am trying to understand this, so that I can correct my mental model of Rust, lifetimes, and the borrow checker. But the only answers I've found so far are akin to this one: simply saying it can't be done and ending the conversation without actually convincing me.

Consider a function like:

fn find_word_in_text(
    text: &str,
    keywords: &[&str],
    options: &MatchOptions,
) -> &str {...}

Today, this does not compile, and requires an explicit annotation.

fn find_word_in_text<'a>(
    text: &'a str,
    keywords: &[&str],
    options: &MatchOptions,
) -> &'a str {...}

In the world you propose, this function would compile — but it would have an overly restrictive output lifetime. Users not familiar in depth with the lifetime system might then write callers to obey that restriction (carefully making sure that the options lives as long as text), and even worse, write further explicit lifetime annotations that are following this constraint. This could go down a deep hole of following what seem to be the compiler’s requirements, and producing a program that can't actually satisfy reasonable use cases like generating options just before calling the function.

Requiring the user to specify what they meant more often reduces (but does not eliminate, of course) the chances that they will get stuck in a local minimum of bad default assumptions. “More functions successfully compile by default” is not always a benefit.

8 Likes

So the issue is not that elision would yield incorrect programs on its own, but that developers, upon encountering an error, are more likely to adjust their method caller than the method they are calling - potentially leading to cascading bad code?

  1. it wouldn't be a very usefull default
  2. the errors when it stops working will be more confusing - did I accidentally break the bound and need to fix the body, or do I need to spell out the new bounds?
  3. it's an arbitrary default, that might not fit everyone's mental model
  4. you can't even express "whichever is shorter" with manual annotations at the moment
1 Like

I don't know if this is accurate.

fn shortest_lifetime<'a>(one: &'a str, two: &'a str) -> &'a str {
    // ...
}

When checking this, the borrow checker will ensure that the returned value is valid while both inputs one and two are valid. In other words, the moment one isn't valid, the result is no longer valid. In other words, the moment the "shorter" lifetime wasn't valid, the result was no longer valid. In other words, the result had a lifetime equal to the shorter lifetime.

As @kpreid pointed out:

Yes, but in general, lifetimes whether explicit or elided can never “yield incorrect programs”, because lifetimes don't change the run-time behavior of the program. Any given choice of lifetime annotations on a function results in a function with a certain set of constraints on the input, and those constraints can be:

  1. ideal for the actual problem,
  2. over-constraining intentionally to allow future evolution,
  3. over-constraining but in a way that is unavoidable due to expressiveness limitations,
  4. over-constraining in a flawed way that can be avoided with better annotations,
  5. over-constraining in a way that makes the function unusable, or
  6. under-constraining in a way that prevents the function itself from compiling (it needs to do something that the signature does not permit).

Most of these are unavoidable. The thing we really, really have to be careful about is lifetime elision creating case 4 — cases where writing the program naively produces a constraint that is problematic.

Well, yes, but that would be in conflict with the elision rule for functions that take multiple references and don't return any.

Note that lifetimes never change behaviour, they just reflect a truth. So you can add literally any elision rule for lifetimes and there's still no situation where the borrow checker fails to catch an error.

I think the core piece here is that the "well it's just the shortest" is nice for the body of the function with only & references, but not for the caller. Because now you've tied the lifetime of the return value to both inputs. And that means that if you have the "haystack and needle" case, you're borrowing from both which is wrong, and constraining what you can do in the caller unnecessarily. And if they're both invariant in the lifetime, then it's not "shortest" it's "exactly the same", which is even more of a hazard.

So to me it's like how we don't elide fn foo() -> &str; either, because we want to give you the error saying "hey, that's weird -- you probably want String, but if not then you have to say &'static".


The other trick here is that we often just don't write the references in places where it's the same. Look at cmp::min, for example: it does exactly the "return the shorter lifetime", but it does that by not mentioning references or lifetimes at all. It's just T, T -> T, and if you instantiate that with a reference then great.

That's often a really nice way to not need lifetimes explicitly, especially in conjunction with impl<T: Foo> Foo for &Foo forwarding impls.

2 Likes

This is probably the subject of a new post, but I just took a look at cmp::min and I'm now incredibly confused how this method even works...

It seems like it's taking ownership of its parameters since it takes a type T and not a borrowed reference &T. Yet somehow I can pass references to this method and it still works!

fn main() {
    let a = String::from("Hello");
    let b = String::from("World");
    {
        // You can pass the Strings directly, and ownership is taken:
        // let result = std::cmp::min(a, b);

        // But you can also pass references (somehow)!
        let result = std::cmp::min(&a, &b);
    }
    // Because we don't give up ownership, this works fine:
    println!("{} {}", a, b);
}

What is this black magic?

The signature of std::cmp::min is:

pub fn min<T: Ord>(v1: T, v2: T) -> T

And the standard library declares a blanket implementation of Ord for references:

impl<A: Ord + ?Sized> Ord for &A { ... }

So since String: Ord, &String: Ord as well, and we can call min with T == &String:

let result = std::cmp::min::<&String>(&a, &b);
4 Likes

This is a very important general principle: reference types are types. Any time you see a type variable T, T might be equal to &U unless some bound on the type variable prohibits it. A lot of Rust’s conveniences and flexibility depend on this.

11 Likes

My 2 cents: Hylo

This is why I mentioned

Often a great way to take a &'a mut impl Iterator as a parameter is just to take impl Iterator as a parameter, since &mut impl Iterator implements Iterator.

Note that using the same lifetime can cause very weird errors if some of the types involves are invariant. For example a function like fn foo<'a>(bar: &'a mut Bar<'a>) is very difficult to call without an error (if not impossible depending on how Bar is defined). Instead, it's generally better to have different lifetimes for every input, e.g. fn foo<'a, 'b>(bar: &'a mut Bar<'b>).