I have presented here a simple proposal that tries to address the "nested method call" problem as part of the NLL work, without modifying the desugaring into MIR at all (or changing MIR's dynamic semantics). It works by augmenting the borrow checker so that mutable borrows begin as "reserved" and then, on first use, convert to active status. While the borrows are reserved, they impose the same restrictions as a shared borrow.
I keep seeing the phrasing âto use a mutable borrow,â but itâs not clear to me what that means (which is bothersome because it seems to be the crux of the dilema!). Is there a precise meaning for this? E.g. at the boundary of a function that takes it as an argument? Or maybe when it is dereferenced?
let x = 3;
let p = &mut x;
let q1 = p; // Is this a "use" of p?
let q2 = *p; // How about this?
let q3 = &mut *p; // Eh?
Hm, and itâs always bothered me that closures mutably borrow their environment immediatelyâŚ
let mut vec = vec![];
let mut f = |i| vec.push(i); // is creating this closure a use of &mut vec?
f(vec.len());
This focuses on the case where the âreservedâ phase comes strictly before the âmutably borrowedâ phase. Or, in the terms of alternative âtwo-phase lifetimeâ proposal, where the write lifetime is a suffix of the read lifetime. But thereâs also an interesting case where the âwriteâ lifetime is a prefix of the âreadâ lifetime. This is detailed in the Rustonomicon and I have seen it several times in real-world code:
struct Foo(u32);
impl Foo {
fn mutate_and_share(&mut self) -> &u32 {
self.0 = 5;
// `self` will no longer be mutated after this point
&self.0
}
fn share(&self) {}
}
fn main() {
let mut foo = Foo(1);
let loan = foo.mutate_and_share();
foo.share();
}
It seems like this could be solved as part of the same mechanism, as long as a mutable borrow can be âdeactivatedâ as well as âactivatedâ. This could be considered âthree-phase borrowingâ since there would be a âreservedâ phase, then a âmutably borrowedâ phase, then a âsharedâ phase. Alternately, it could be implemented as âtwo-phase lifetimesâ where the âwrite lifetimeâ is any continuous subset of the âread lifetimeâ (not necessarily a suffix).
Heh, I wondered if someone would call me on that. I did indeed have a precise meaning in mind. I meant that it appears in MIR in some position other than being directly assigned to (often called a "use" in compilers, as opposed to x = ..., which is a "def").
So, in answer to your questions, the answer in all cases is yes:
let x = 3;
let p = &mut x;
let q1 = p; // Is this a "use" of p? YES
let q2 = *p; // How about this? YES
let q3 = &mut *p; // Eh? YES
...
let mut vec = vec![];
let mut f = |i| vec.push(i); // is creating this closure a use of &mut vec? YES
f(vec.len());
Basically, anytime that the variable name appears but is not being directly overwritten.
Perhaps. I think the tricky part here how to communicate to the caller that itâs ok for them to allow the receiver to be mutated after mutate_and_share() returns. In particular, the question is this: what happens to the state that you did not return. Can mutate_and_share() assume that nobody will be touching that state? Right now, it can assume that, though only for the lifetime 'a. Given that mutate_and_share() canât rely on destructors to run, Iâm not sure if there is a profitable way to take advantage of that assumption. Weâd also have to specify precisely what the rules are for when we decide that the âmain borrowâ is âdeactivatedâ. Is it ok if the &u32 is returned embedded in a struct (e.g., Slice<'a>)? If so, how do we know that itâs ok? Seems like we have to look at the struct fields (perhaps we can deduce it from variance).
Anyway, it seems like a complex topic to me. I know there have been several threads on it.
If you mean whether it should be able to assume that, then surely it should be able to assume the state won't be written/mutably borrowed, since the same goes for regular &self -> &u32, but may be read/immutably borrowed, as the goal is to call &self methods afterward.
But after thinking about it, I guess you're worried about what unsafe code may currently be assuming?
I came up with an example. It's pretty nonsensical and unlikely, but I believe the current borrow rules guarantee it to be safe, whereas it would be unsafe if &'a mut self -> StructContainingImmutableRef<'a> allowed future immutable borrows of self. Basically, it involves a union with a safe method from &self to variant A, which normally always contains A, except for a method that takes &mut self, swaps the variant to B, and returns a struct wrapping an immutable reference to that.
We could consider reserved borrows to be something akin to the old const borrows we used to support: these would permit reads and writes of the original path, but not moves. There are some tricky cases to be careful of (for example, if you reserve *b where b: Box, you cannot permit people to mutate b, because that would cause the existing value to be dropped and hence invalidate your existing reference to *b), but it seems like there is nothing fundamentally stopping us. I did not propose this because (a) I would prefer not to introduce a third class of borrow restrictions and (b) most examples which would benefit from this change seem quite artificial and not entirely desirable (though there are exceptions). Basically, it seems ok for vec.get({vec.push(2); 0}) to be an error. =)
I had a thought that if a third class of borrow restriction were not only added (scary) but exposed in the type system (much scarier), it could make some helper methods more ergonomic - ones that look like:
Right now, calling that locks you out of the whole rest of self, because the contract is "return value is alive and won't be touched as long as self is alive and won't be touched". In theory, if you could change the signature to have the contract just be "return value is alive as long as self is alive", and then later "activate" it into a mutable borrow (which would still lock out all of self, but only temporarily), you could greatly reduce spurious lifetime conflicts.
But that doesn't help if baz is behind a pointer or option or something, because then that contract can't be guaranteed. So this is probably a dumb idea. If the problem needs solving at all, I guess it would be better to approach it with things like "which fields I borrow" specifications on methods (which are really ugly, but at least more flexible).
The v.push(v.len()) thing makes me wonder. Are such nested function calls codified? Someone on /r/rust mentions that they don't understand why it's not desugared to
let foo = v.len();v.push(foo)
which is good enough in that simple case, but what about more complex things, like
I think that if you start allowing âyet-to-be-borrowedâ lifetimes to be present in types, you canât rely on the borrow-checker to enforce them, because of things like:
let mut x = 1;
let mut y = 2;
let mut x_ = (&mut x,);
let mut y_ = (&mut y,);
mem::swap(&mut x_, &mut y_);
// must allow (one occurrence) of these lines in both orders, but
// not concurrently
read(*y);
*x_ = 2;
In the ânormalâ case, the âlifetime as set of pointsâ proposal dually identifies a live lifetime with itâs set of exits. I suppose we can allow for âyet-to-be-liveâ lifetimes by identifying them with their set of starts and set of exits (and therefore, in the dual âset of pointsâ representation, forcing all points between a start and an exit to be live). We can achieve this in inference by adding dataflow equalities that force regions to be contiguous.
Also, LSP computes the âpreciseâ lifetimes of each borrow by itself - even if you have eqtys, 'a ~ 'b is defined as 'a <: 'b ⧠'b <: 'a - LSP subverts eqty, so there is no need for borrowck to do its own dataflow.
What I am wondering is whether it can assume that the state will not be read. To step back, when you see a signature like this one:
fn foo<'a>(&'a mut Foo) -> &'a u32
The way I think of it is: you give me exclusive access to some memory containing Foo for the lifetime 'a. I give you back shared access to some memory containing a u32 for the remainder of the lifetime 'a.
There is no a priori reason to think that (a) the u32 is part of the Foo or, even if it is, that (b) I have relinquished my exclusive access to the other parts of Foo.
Now, it may be that one cannot make productive use of those other parts of Foo. For example, if I had (e.g.) started a thread that was accessing the other fields of Foo in parallel, I wouldn't have any way to make that thread stop executing before 'a concluded -- but this argument is subtle, and it relies on the fact that 'a is bound on the method (it's a kind of lifetime parametricity argument).
So I guess what I'm saying is that I can imagine yes that we could make some rules for when the caller must be "relinquishing" their borrow (or at least converting it to a shared borrow) but I'm wary and would like to think more deeply about it. At best that would be a sort of "pattern matching" on the signature, since we don't currently have a way to express that concept in the type system. (Which might be satisfactory.)
Regardless, this is a good example to keep in mind. I've been meaning to write up a post going through my list of "borrowck challenges" -- I think that NLL and the nested method thing addresses a good many of them, but certainly some remain. My personal list I just put into a gist with some very brief notes, but it's probably incomplete.
This is an interesting example. To be clear, the system I was thinking of would consider the mutable borrows of x and y to start immediately, since (in the MIR) the temporary would get used when it is packaged up into a tuple.
If we were just trying to track mutation regions, if seems like the mem::swap line would require that the type of x_ and y_ be equal (at that point), which would seem to imply that their write regions would overlap (at that point). This would then imply that the two borrows are overlapping and an error would be reported, right?
That is, I wouldn't expect this program to be legal, as long as any write occurs (since otherwise the write regions would be empty).
This is what I referred to as âchanging the desugaringâ. The previous internals thread did indeed discuss various options like that, along with their pros and cons.
I wanted to expand on my answer. I think I overstated the case in my conclusions and comparisons about the "equivalence" between this borrowck-based approach and a type-based approach (even one that guarantees continuous regions). In particular, I intentionally adopted a limited and simple version of "use" (i.e., one that cannot "see through" a move, or embedding the value into a structure). Most likely, if we took a type-based approach, you would get a similar feel, but with a richer notion of "use".
Example:
let a = &mut foo;
let b = a; // alternative: `let b = (a, );`
// is the borrow of `foo` active here?
With a type-based approach, presumably the type of b would be equal to the type of a (or, in the alternative case, it would be a tuple that includes a). I think that if we wanted a richer notion of "use" like this, I'd probably pursue some kind of type-based alternative -- you probably could model it with the borrowck, at least those examples, but there are a host of others where it might not work (e.g., let b = vec![a];).
That said, I don't really feel that's necessary. I think the main goal is to support the simple example of vec.push(vec.len()), not to make a super general concept of "delayed activation" that people will use all the time. I think the simple definition I gave is also readily explainable. When you do let x = &mut..., the next time you refer to x (basically), it is activated. When you do foo(&mut) or some such thing, it is as if you did let tmp = &mut; foo(tmp), and hence it is activated. But I have to ponder it -- perhaps it winds up being confusing in practice.
Yup. The advantage of true type-based approaches is that they work well with functions. And I want to pass multiphase borrows to functions because that solves at least the get/get_mut problem and allows writing the @comexfoo function.
On the other thread, I think that discontiguous borrows are an inherent problem of may_dangle lifetimes:
struct Dangling<#[may_dangle] 'r> {
data: *const u32,
lifetime: PhantomDanglingLifetime<'r>,
}
fn main() {
let mut x = None;
loop {
let a = 0;
x = Some(Dangling::new(&a));
}
}
The reason is that liveness does a very good job in preventing âlifetime use-after-freeâ, but if we allow dead lifetimes that guarantee is gone - so we need a special category for ânon-freed but inactiveâ lifetimes, and we probably want to bound their lives with another lifetime.
No unsafe code, fixed or invariant lifetimes, or structure returns, just (&mut self) -> &T, but calling two with the borrow from one still active would be Bad.
