Having mutability in several views of a struct

[nikomatsakis]

Changing this to non-trivial case:

So, first, hold-up. I was very specific in my initial example to use private functions: the majority of time that I hit this problem, it concerns internal details of a type. Basically, I would prefer to have some helper functions, but I cannot, because the borrow checker doesn’t let me. This – the private function case – is I think a distinct use case worth considering separately when it comes to ergonomics (maybe not the underlying mechanism).

I find that when I am working with types “externally”, as in your example, it is much more unusual for me to have the insight into “what fields a method may touch”. I tend to treat it as an atomic unit, and I don’t find the rules prohibitive (now, maybe it influenced the author of the type in terms of the public API that they export).

There are some exceptions: the main ones that I’ve noticed involve collections. For example, I like split_at_mut on vectors, which covers more-or-less this same case. Similarly, I sometimes want the ability to have multiple mutable refs into a hashmap.

All of this is not to say that we don’t need to worry about external users at all – but I would say that for an external user, defining an explicit view or trait is not necessarily prohibitive. It makes sense because it is giving your users a “window” into your “mental model”. i.e., if you have a struct Connection that exposes two views, Buffer and Address, then you are telling them they can think of a connection has having those subparts, and then you may have distinct methods that sometimes operate only on the buffer, sometimes only on the address, etc.

Anyway, I am pretty excited about the idea of leveraging fields in traits for this. (Don’t have time to write a bunch here, more later.)

[nikomatsakis]

This – the private function case – is I think a distinct use case worth considering separately when it comes to ergonomics (maybe not the underlying mechanism).

Re-reading this, I don’t think it fully communicated my intent. First, if possible, I certainly prefer a mechanism that acts the same for internal/external cases. Leaning on privacy always feels like a bit of a smell.

But secondly, part of what I meant to say here was that it’s very important that you be able to “fully encapsulate” these views so that they are not visible in your public API. (But I don’t think using traits to describe them is incompatible with that.)

(But circling back to the first point, if we were going to try and make something work semi-automatically – e.g., inferring views – I would definitely limit it to the private fn case. But I’m happy to shelve all talk of inference for now and focus on explicit notation. I suspect that would not be too onerous, this doesn’t arise that frequently.)

[BatmanAoD]

(Is this the right place to woodshed possible approaches?)

I think having a new contextual keyword would be good here; my suggestion is to treat views as an alternative type of reference for self (and, down the road, perhaps for other arguments). So, we would introduce a new type of self parameter for methods:

fn takes_view(&view self)
{
    self.a = 7;
    println!(self.b);
}

Here, view is the conditional keyword, referring to a “view-type”, which would be a compiler-generated struct of the form described above (where the members are all references to members of another struct). Use of &view would require the compiler to:

1. Determine which fields in self are mutated in the function (in this case, a)
2. Determine which fields are read (in this case, b)
3. Generate the correct view-type for that combination of field-references
4. (optional) Merge identical view-types
5. Replace &view self with a mutable (if necessary) reference-to-view-type argument

At the call site, the view-type would be automatically instantiated and passed as the first argument.

[Soni]

my proposed PIT syntax also ties in with partial borrows.

[H2CO3]

Not sure if this needs a new keyword. Can’t borrowck can be taught to carry partial borrow information through reference types themselves? That would mean that the language surface wouldn’t change, it’s just that the borrowing mechanism would become smarter and allow more valid code, in a way similar to the transition to non-lexical lifetimes.

[Yato]

This shouldn’t happen over a public interface otherwise simply changing the implementation of a function becomes a breaking change. I would even limit it to work only within a module, otherwise there is too much action at a distance, which is bad.

[Soni]

Wouldn’t the issues of public interface and stuff be solved with PIT syntax? (we don’t need full on PITs, we just need the syntax)

E.g., standard PIT stuff:

1. In public interfaces, PIT fields must be public:

   pub fn foo(&Foo(t)) -> &Bar {...} // compiles only if `struct Foo { pub t: ... }`
   

   Basically fields must be visible everywhere the function/etc is visible.
2. PITs can only be downgraded, i.e. this is invalid:

   struct Foo { a: ..., b: ... }
   struct Bar { a: &Foo }
   fn f(foo: &Foo(a)) -> &Bar {
       Bar { a: foo }
   }
   

   because Bar wants a full Foo whereas f only has a partial Foo.
3. All the stuff about PIT references, really.

But we should have the following differences:

1. No actual PITs. These would only be valid for references. No let foo: Foo(t), only let foo: &Foo(t) and let foo: &mut Foo(t).
2. No initialization stuff either (&mut Foo(a) -> Foo(b)).

[BatmanAoD]

Not sure if this needs a new keyword. Can’t borrowck can be taught to carry partial borrow information through reference types themselves?

This shouldn’t happen over a public interface otherwise simply changing the implementation of a function becomes a breaking change.

Well, that’s part of why I suggested the &view syntax/keyword. It’s a balance between being explicit and permitting the compiler to do the grunt-work. Views in public functions are part of the public interface, yes, but restricting them doesn’t seem entirely reasonable, nor does forcing the user to manually define view structs.

[BatmanAoD]

Wouldn’t the issues of public interface and stuff be solved with PIT syntax ?

Stupid question: what’s a PIT?

[Soni]

Partially Initialized Types, it was an idea that would unify proposals for uninit references and opaque references (thankfully neither of which because a thing!) and I guess (now) also partial borrows.

it’s rather explicit, because it’s not allowed to do type inference across function boundaries. (i.e. types must be specified in function declarations.) but that actually avoids some misborrowing bugs that can happen sometimes.

[Yato]

Here’s a link to @Soni’s Pre-RFC for PIT.

[Pre-RFC] Partially Initialized Types

there’s been a lot of talk about &uninit references, MaybeUninit, placement-new… I’d like to propose an alternative to all of them: Partially Initialized Types. They’re, arguably, the cleanest approach to all of this. Consider types: struct Foo { a: i32, b: i32 } struct Bar { a: i32, b: Foo } Then, we can have: let x: Bar(b(a)) = Bar { b: Foo { a: 1 } }; However, passing these to functions is mostly pointless: fn foo(x: &mut Bar()) { // can't do anything with x here } foo(x); // but c…

[Yato]

Does view specify which fields are being referenced and what mutability they have, if not then it is still action at a distance. @Soni’s PIT syntax almost does it, if mutability was specified per field but the syntax is almost as bad as just creating a new struct (and is even worse if there are mutiple functions that take the same fields with the same mutabilites).

---

I personally don’t think this proposal can carry its weight, I think a decent solution would create too much noise and add too much weight to the language to be sufficiently useful. I also have never ran into this problem and had a hard time solving it. Normally splitting the data into multiple related structs does the trick.

Also if this sort of thing applies to a public interface, I think it would make it very hard to change private implementation details if needed. I find this to be a bad thing. Therefore, if at all, this should only be limited to crate boundaries.

---

This also seems like something a proc macro could solve. A proc macro could create all the view structs that you need. You would just need to specify which ones you need, then you could use the types that the proc macro generates.

[H2CO3]

Oh yes, that is right. But that in fact applies to any implementation / syntax which tries to infer the borrowed fields from the implementation of the function. So merely the view keyword wouldn’t help here.

I could imagine adding annotations of partial borrowing to reference-type function arguments instead. Something along the lines of fn foo(&self::{ bar, baz }) { … } indicating that foo only borrows self.bar and self.baz (which would then be enforced within the function body, and not inferred.)

(Oh my gosh that syntax looks really ugly btw, just wanted to come up with something quickly.)

[Soni]

(Oh my gosh that syntax looks really ugly btw, just wanted to come up with something quickly.)

fn foo(&self(bar, baz), other: &Self(bar, baz))

(from PIT thread)

[dan_t]

One could also mimic the field access syntax:

fn foo(&self.bar, other: &Self.bar)

And for multiple fields the ‘use’ syntax:

fn foo(&self.{bar, baz}, other: &Self.{bar, baz})

[Ixrec]

One thing I’m not super clear on when it comes to this whole inter-procedural borrow checker interference with disjoint self borrows issue:

Does this typically happen when the fields being borrowed are already public? Or are they typically private?

Because if they’re private, then fn foo(&self.{bar, baz}) is solving the problem by putting private fields into your public API signatures. But if they’re already public, then it seems worth the hassle of breaking up the struct into multiple public types to make an explicit commitment to their disjointness.

[H2CO3]

…which in turn makes me think: why don’t we just pass those fields as separate references in a similar situation? Then it wouldn’t require global reasoning (nor any modification of the language).

[Soni]

the PIT thread also proposed something interesting: tuple structs leak implementation details, as such maybe we should too.

[BatmanAoD]

Does view specify which fields are being referenced and what mutability they have, if not then it is still action at a distance.

No; I was trying to come up with a syntax that would fulfill @nikomatsakis’s “idea that it would be nice… for the compiler to do the inference for you.”

I also don’t quite agree that this is “action at a distance”, since the set of fields included in the view is determined where the view keyword is used. But you’re right that when used in a public API, the permissible calling contexts would depend on the function implementation, so I see why you’d use that phrase.

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.