[Idea] Two lifetimes on unique references

[adamAndMath]

I've seen some conversations about wanting to downgrade &mut to a & through a function, in such a way that the original value can still be referenced.

struct Foo<T>(T, usize);

impl<T> Foo<T> {
    pub fn new(val: T) -> Self {
        Foo(val, 0)
    }

    pub fn inner(&mut self) -> &T {
        self.1 += 1;
        &self.0
    }
}

The problem is that the unique reference must live as long as the returned shared reference. To solve this we don't need the shared reference to outlive the unique reference, but rather that the reference outlives it's uniqueness.

This leads to the conclusion that unique references could be extended to have 2 lifetimes. One for how long the reference lives, and one for the time in which it is unique.

I'm proposing the syntax &'a mut'b T where 'a is the references lifetime, while 'b is the lifetime of uniqueness. Further more I'll propose that both &mut and &'a mut desugars to &'a mut'a, making the change backwards compatible. This would also mean that this is completely opt in. So as to fix the method I'd have to write

fn inner<'a, 'b>(&'a mut'b self) -> &'a T

I'm not sure whether this is something I want in the language, but it seemed like a solution to a problem, so I thought I should share it.

[RustyYato]

This seems really complex, and I'm not sure how this will generalize. To solve this problem, what you can do is return &Self and &T from your function, then you can use the returned reference later.

struct Foo<T>(T, usize);

impl<T> Foo<T> {
    pub fn new(val: T) -> Self {
        Foo(val, 0)
    }

    pub fn inner(&mut self) -> (&Self, &T) {
        self.1 += 1;
        (self, &self.0)
    }
}

[adamAndMath]

@RustyYato

This seems really complex, and I'm not sure how this will generalize.

Could you explain what you mean. I'm not sure in what direction to generalize, and it's often hard to see the complexity of once own ideas

[RustyYato]

@adamAndMath

often hard to see the complexity of once own ideas

Lifetimes are already a tricky subject for new user of Rust, so any added complexity should be carefully reviewed to see if it is worth it. In this case, a whole new lifetime parameter for minor convenience is certainly not worth it.

More over it is unclear what you mean by "lifetime of uniqueness" and "lifetime of reference", and what's the difference between them.

@adamAndMath

I'm not sure in what direction to generalize

That was my bad, didn't mean to put that in, I was thinking of something else while writing that.

[Ixrec]

I think this old post by @nikomatsakis is still a fairly accurate summary of where we're at with this sort of idea:

Relaxing the borrow checker for fn (&mut self) -> &T

So this issue comes up repeatedly. My take is this: In the current model, @kvijayan’s claims are not quite correct, but the issue he identifies is annoying, and it’d be nice to have a better solution. First let me explain the model as I see it. I think this is quite similar to what @eddyb has been saying, though there are a lot of comments here and I’ve not parsed them all with 100% care. I think it’s easiest to think of the current model in terms of permissions. So when you do the borrow you g…

Probably the biggest change since he wrote that is simply that we have NLL now , but "a proposal for a Rust memory model" is still a ways off (see GitHub - rust-lang/unsafe-code-guidelines: Forum for discussion about what unsafe code can and can't do)

[mcy]

@RustyYato

More over it is unclear what you mean by "lifetime of uniqueness" and "lifetime of reference", and what's the difference between them.

For what its worth, I often feel that the way we teach references is suboptimal. I feel like "sharedness" and "uniqueness" aren't quite as dual as "const" and "mut", and I think that the idea that a reference can outlive its uniqueness is an interesting way to think about reborrowing.

I guess the way this could be confusing is how you might infer &'a mut'?0 T. Do we infer ?0 separately or do we always infer 'a unless specified by the user? The latter feels like it lines up better with what we expect of &mut T, if anything. I'd definitely class such a feature similar to 'a: 'b and for<'r>: advanced lifetime features that are necessary but not a core part of the mental model.

[RustyYato]

@mcy

For what its worth, I often feel that the way we teach references is suboptimal. I feel like "sharedness" and "uniqueness" aren't quite as dual as "const" and "mut"

I completely agree, I think we should start the process of changing all official documentation do use uniqueness instead of mutability. I have yet to meet someone (here, users forum, or on reddit) that doesn't understand uniqueness, so I think that this change will make Rust more accessible and easier to learn.

@mcy

and I think that the idea that a reference can outlive its uniqueness is an interesting way to think about reborrowing.

I don't think that this needs to be encoded into the types, especially when it is really easy to also return self if you want to allow shared usage of self along with the unique borrow.

[mcy]

@RustyYato

I don't think that this needs to be encoded into the types, especially when it is really easy to also return self if you want to allow shared usage of self along with the unique borrow.

I'm not sure. I've never been a big fan of the Rust pattern of "maybe take ownership by taking a T and maybe returning a T". I'm not sure if it should be encoded in the &mut syntax, but decoupling "I have a reference inside 'r" from "I have unique access inside 'u" as a first-class concept is valuable to me.

[RustyYato]

While yes, I agree that is valuable, I don't know if that is worth adding a whole new lifetime parameter. Given that lifetime parameters are already very confusing for newcomers, adding another may slow down Rust's growth too much.

How often does this come up for you? Personally this doesn't come up for me that often, so I don't see the value of adding a new lifetime parameter for what seems like minor convenience.

[mcy]

I've mostly only encountered this sort of thing with data structures? For example, I have a vector-like thing where I can push in a T and, if that succeeds, get a &T which points into the data structure. Unfortunately, such a function is fn<'a>(&'a mut self) -> &'a T, so the borrows in 'a include a unique borrow to self, so I can't do

let t = ts.push(t);
let len = ts.len();

I've had similar problems with wanting a map of boxes to sort-of emulate std::unordered_map...

[Ixrec]

@mcy

I've mostly only encountered this sort of thing with data structures?

Agreed. I think https://github.com/rust-lang/rfcs/pull/2025 helps with a lot of these cases in practice (after all, the standard motivating example is vec.push(vec.len());).

Interestingly, that thread explicitly proposes references having two lifetimes, but that appears to be borrow checker implementation / inference algorithm details and not anything close to a surface syntax proposal that users would be expected to deal with.

So although that's clearly not exactly what's being asking for in this thread, I strongly suspect that that already gets us most of the way to the optimal complexity/convenience trade-off, by making Rust just accept more code without forcing any new complexity on the user.

[adamAndMath]

@Ixrec

I think https://github.com/rust-lang/rfcs/pull/2025 helps with a lot of these cases in practice (after all, the standard motivating example is vec.push(vec.len()); ).

That rfc does have a lot in common with my proposal, though it's about delaying unique access. I don't see any way to expose this in user land, as one of the lifetimes is shorter than the variables lifetime.

Further more, it can't solve the problem in question, as these lifetimes do live in userland. Thus any solution must enable naming of such lifetimes. If not, it would make elision of lifetimes able to describe something more than explicit lifetimes.

---

@mcy

For what its worth, I often feel that the way we teach references is suboptimal. I feel like "sharedness" and "uniqueness" aren't quite as dual as "const" and "mut", and I think that the idea that a reference can outlive its uniqueness is an interesting way to think about reborrowing.

It is interesting that "const/mut" leads to ideas like write-only references, while "shared/unique" lead me to this idea. Either doesn't make much sense in the other paradigm.

[CAD97]

Just as a real-world example where "unique deterioration" would be useful, HashSet::get_or_insert is defined as fn get_or_insert(&'_ mut self, value: T) -> &'_ T. Instead, with this, it could be fn get_or_insert<'a>(&'_ mut'a self, value: T) -> &'_ T.

[mcy]

I think this is equivalent to my example; I'd be interested to hear about different cases in which you want something like this.

@adamAndMath

It is interesting that "const/mut" leads to ideas like write-only references, while "shared/unique" lead me to this idea. Either doesn't make much sense in the other paradigm.

I don't think that's quite true, actually! The only reason to want write-only references is to be able to have references that temporarily point at unitialized memory. I think there is a reasonable mental model (though one which does not surface in the syntax perhaps) of various properties of a reference having different lifetimes:

• Allocated-ness: during some lifetime, this reference points to memory that we have some kind of access to (hopefully someone like @RalfJung can specify the notion I want here, I believe that it is a weaker form of "validity" in the UWG sense). Addresses like 0x0, for example, has alloc for lifetime '![1] while data embedded into the binary for lifetime 'static; stack and heap memory are alloc for some reference 'a.
• Initialized-ness: during some lifetime, this reference points to memory that is known to be initialized. Currently, &'a T implies intialized-ness for 'a while a theoretical &'a uninit might indicate initialized-ness for some unknown lifetime such as '!. To be read, a reference must be within a lifetime for which it is initialized.
• Uniqueness: during some lifetime, this reference is the sole reference from which a particular memory region is reachable. Currently, &'a mut T implies uniqueness (and initialized-ness) for 'a; a theoretical &'a uninit T merely implies uniqueness. To be written to, a reference must be within a lifetime for which it is unique (pretend there is no such thing as const statics for a moment).

If we were going to write out a totally explicit reference from which the other reference types can be derived, you might write &'a init'i uniq'u T[2], and define

&'a T        := &'a init'a uniq'! T;
&'a mut T    := &'a init'a uniq'a T;
&'a uninit T := &'a init'! uniq'a T;

(Debatably, we could also define raw pointers as &'! init'! uniq'! T.)

[1] '! is the "empty" region or "dead" lifetime, opposite to 'static. This has no meaning in actual Rust, since to witness a region 'a one must be within it.

[2] To be clear, we do not want to surface such insane syntax.

[adamAndMath]

@mcy

To be clear, we do not want to surface such insane syntax.

To be extra clear, we could not surface such syntax, as these references live longer than the lifetimes they contain. I do however think there is value in thinking of the properties in such a way.

[Tom-Phinney]

@mcy

I don't think that's quite true, actually! The only reason to want write-only references is to be able to have references that temporarily point at unitialized memory.

To quote you, "I don't think that's quite true, actually!" There are embedded processors with peripherals that have some memory-mapped write-only registers. When there is more than one instance of such a peripheral, or the base address of the peripheral register block is configurable, it is useful to access the peripheral's registers by reference.

[mcy]

@Tom-Phinney

peripherals that have some memory-mapped write-only registers.

You do not want references when dealing with MMIO in any situation, because LLVM can reorder reads and writes, and also speculatively read references. If you have an MMIO device where each sequential read triggers an update of the underlying register, you're gonna have a bad day.

You want to build a safe abstraction on top of raw pointers and volatile reads and writes. These are guaranteed to commit real reads and writes in true sequential order on every platform, or your money back. This is similar to how interacting with such memory via char* in C/C++ will have the wrong semantics, and should instead be declared volatile char*. Back when I wrote a toy operating system in C, a lot of the fussing about with the disk driver required volatile memory to work correctly wrt the memory model.

[Tom-Phinney]

You are correct that all such memory-mapped I/O has to be volatile. If references as mapped to LLVM can't support that mode of access, then the access needs to use other functionality.