[Pre-RFC] `field_projection`

language design
21

Maybe I’m misunderstanding the proposal here, but isn’t it determined by the field name in Pair? For example:

let x: Option<&Pair<String,String>> = …;
let key: Option<&String> = x.key;
let val: Option<&String> = x.val;
22

I am talking about projecting the inner type: Pair<&Struct, String> -> Pair<&Field, String>.

Your example already works when Option becomes #[inner_projecting].

When marking Pair with #[inner_projecting] one would allow

struct Point { x: u32, y: u32 }
let p: Pair<&Point, String> = …;
let x: Pair<&u32, String> = &p.x;
let y: Pair<&u32, String> = &p.y;
23

If you now have

struct Point { x: u32, y: u32 }
let p: Pair<&Point, &Point> = …;
// we could allow projecting both at the same time
let x: Pair<&u32, &u32> = &p.x;
// or specify which fields, I do not like this, hence I suggested only allowing single generic params
let _: Pair<&u32, &Point> = &p.{key}.x;
let _: Pair<&u32, &u32> = &p.{key, val}.x;
let _: Pair<&Point, &u32> = &p.{val}.x;
24

Ooo, this reminded me! One use-case I've run into is something that one would typically use rental for -- something that both owns its data and has pointers into its data -- imagine a String coupled with a data structure storing slices into that string. This works great if it's all in a single data structure, but as soon as you want slices to be in disparate independent data structures and don't want to propagate borrows, it's nice to have an Rc<str> that allows projecting into an RcSlice<str> (like https://crates.io/crates/rc_slice, but ideally storing a direct reference to the projected content, like RefCell's Ref and RefMut projections).

That being said, I'm not specifically advocating for field notation to do this projection. Honestly, even a method in std would be lovely.

25

This makes the trait more complex but I don't think it's that bad. It enables projecting existing Arcs without changing the definition of the type, which is a huge win

26

That is an interesting use case, I also encountered the "I want to store data in an Arc/Rc, but also want pointers into it without lifetimes" problem. I think that an Projected{Rc, Arc} does not seem too far fetched. I think it could look like this:

pub struct ProjectedRc<T, S> {
    backing: Rc<S>,
    ptr: NonNull<T>,
}

impl<T> Rc<T> {
    pub fn project<U>(&self, map: impl FnOnce(&T) -> &U) -> ProjectedRc<U, T> {
        ProjectedRc {
            backing: self.clone(),
            ptr: NonNull::from(map(&**self)),
        }
    }
}

Not sure if this is sound (if someone inputs a malicious map closure). Also not sure if the type parameters really need to be Sized.

I do not think that it is possible to project Arcs without adding any additional information. I do not like that this will enable implicit Arc creation. This has precedent, for example here is a post that tried to allow cloning Arcs directly into closures. I do not like that foo.bar.baz.bizz can now create possibly 3 new Arcs when someone inevitably refactors one of the fields to be of that type. I want struct.field to be reserved for pointer offsets with possibly zero cost transmute-like operations to adjust the type. I hope this will be better reflected in my next update that I will post soon.

1 Like
27

This is conceptually pretty similar to "optional chaining" in JavaScript, Swift, and a bunch of other languages.

Those languages all have a special operator root?.field which does basically what's being discussed here, but only for null(ish) values (though it isn't restricted to fields). Using ? is probably out, but having an operator like that might alleviate concerns of implicit refcount changes for the Rc/Arc case.

28

I feel like a dedicated function is going to be enough. It is not such a common occurrence compared to field projections with Pin and adding such an operator just for refcount adjustments seems unnecessary. I also think that it would be difficult allowing other behavior for ?., it should be clear what an operator can do. While there is precedent for more flexible operators (I specifically think of + and +=/other arithmetic) with respect to performance, I would not like to see an operator where I have to consult the docs/source code to know if I should use it, or not. A function can be documented in a much better way and also is clearer as to what it does.

All of this is written from a low level perspective that really cares about performance. I know that there are less performance critical situations in which people would like less control and better/easier ergonomics. In this concrete example however, I believe the advantage to be minuscule.

1 Like
29

That's a good point!

Maybe we could have something like project!(foo, bar.baz.bizz), creating a single ProjectedArc?

1 Like
30

That seems like a nice entrypoint for writing a crate. It would be nice to see how the ergonomics/reception is before trying to put it into the stdlib.

31

I made some bigger changes to the RFC. (you can scroll back up and click on the small pen icon in the top right to view the diff, or head over to github to view it there.)

32

In Swift there is something which is possibly more similar: SE-0252: keypath member lookup. I believe it is a combination of this and Deref.

1 Like
33

Personally, I don't really get the desire for Rc or Arc projection. I don't see the benefit over just derefing it.

Also, I'd prefer if projection used a new operator. I'd use -> but that already has a meaning. Maybe ~, ~>, .>, ~., etc.

And for pin projection, could a trait be used instead? I'm envisioning something like this:

unsafe trait PinMutProjectable: PinnedDrop {
    type Projection;
    fn project(self: Pin<&mut Self>) -> Self::Projection;
}

trait PinnedDrop {
    fn drop(self: Pin<&mut Self>);
}

impl<T: PinnedDrop> Drop for T {
    fn drop(&mut self) {
        // SAFETY: because `self` is being dropped, there exists no other reference
        // to it. Thus it will never move, if this function never moves it.
        let this = unsafe { Pin::new_unchecked(self) };
        <Self as PinnedDrop>::drop(this)
    }
}

struct Struct<T, U> {
    pinned: T,
    unpinned: U,
}

struct StructProjection<'s, T, U> {
    pinned: Pin<&'s mut T>,
    unpinned: &'s mut U,
}

unsafe impl<'s, T, U> PinMutProjectable for Struct<T, U> where Self: 's {
    type Projection = StructProjection<'s, T, U>;
    fn project(self: Pin<&mut Self>) -> Self::Projection {
        StructProjection { ... }
    }
}

// also needs a PinnedDrop impl
34

If it existed, it'd allow things like splitting a single allocation into multiple Arc<[T]>s. And doing that is clearly useful, as evidenced by the popular bytes - Rust crate (which of course needs its own type, not a normal Arc<[u8]>).

6 Likes
35

Could you elaborate what your reasoning for this is?

fn foo(struct: &mut Struct) {
    let field: &mut Field = &mut struct.field;
}
// vs
fn foo(struct: Pin<&mut Struct>) {
    let field: Pin<&mut Field> = &mut struct.~field;
}

What you are suggesting is roughly what the pin-project crate does. We could of course merge it into std, but I believe a language feature to be better ergonomically.

36

Here are some important unresolved questions:

  1. for Pin, should we use #[unpin] like other #[inner_projecting], or should we stick with #[pin] (and maybe introduce a way to switch between the two modes).
  2. how special does PinnedDrop need to be? This also ties in with the previous point, with #[pin] it is very easy to warrent a PinnedDrop instead of Drop (that will need to be compiler magic). With #[unpin] I do not really see a way how it could be implemented.
  3. Any new syntax? I am leaning towards NO (except for 4.).
  4. Disambiguate member access could we do something like <struct as MaybeUninit>.value?
  5. Should we expose the NoMetadataPtr to the user?
  6. What types should we also support? I am thinking of PhantomData<&mut T>, because this seems helpful in e.g. macro contexts that want to know the type of a field.
37

I see. I don't use reference counting much, but sounds useful. Perhaps instead of requiring users to use a new Arc/Rc type from the get go, we could have an associated fn protectable(self) -> ProtectableArc that returns a two-pointer Arc/Rc similar to shared_ptr.

38

Well there are a few benefits, I think.

  1. More explicit
  2. No risk of conflicts with union fields
  3. No risk of breaking existing code
  4. I think re-using field syntax for this could be confusing to beginners

It might also support protection for multiple fields better, but I haven't really thought that through.

39

I'm not opposed to adding a macro to more ergonomically support this, but I think a minimal product should just use a trait, or maybe not even support Pin for now.

With the trait, you get to enforce the PinnedDrop requirement for free, and it's very clear that projection is an unsafe thing that needs thought. You could also use the PinProjectable trait as a bound somewhere, though I'm not sure how useful that would really be.

40

I think it is better to be less explicit here. It is easier to port code that uses pin-project, because users will only have to remove .replace() and adjust the type definition.

While this is nice, I would rather want to make the access of non projected fields more explicit. Because the only case where this would happen would be writing a wrapper that supports projection.

Do you have any code that would break under the existing proposal? I thought that I had designed it such that there would be no breakage.

I would argue that &mut T in reality does not have any fields. Only the type T has. We already have field projection for &mut T/&T, we just did not notice :wink:

Do you mean chained projection?

struct MyStruct {
    foo: Foo,
}
struct Foo {
    num: usize,
}
let my_struct: &mut MaybeUninit<MyStruct> = ...;
let num: &mut MaybeUninit<usize> = &mut my_struct.foo.num;

Or do you mean projection of a wrapper type that has multiple projectable fields?

The point is that I am trying to create a safe way to do pin projection. Solving this using unsafe is already possible with the Pin::map_unchecked_mut/Pin::get_unchecked_mut functions. pin-project has widespread usage in crates that want to do pin-projection. Creating an unstable feature with an unsafe trait will not be used, because pin-project provides a better solution...