[Pre-RFC] `field_projection`

language design
1

This pre-RFC has resulted in this RFC.

Summary

The stdlib has wrapper types that impose some restrictions/additional features on the types that are wrapped. For example: MaybeUninit<T> allows T to be partially initialized. These wrapper types also affect the fields of the types. At the moment there is no easy access to these fields. This RFC proposes to add field projection to certain wrapper types from the stdlib:

from to
&MaybeUninit<Struct> &MaybeUninit<Field>
&Cell<Struct> &Cell<Field>
&UnsafeCell<Struct> &UnsafeCell<Field>
Option<&Struct> Option<&Field>
Pin<&Struct> Pin<&Field>
Pin<&MaybeUninit<Struct>> Pin<&MaybeUninit<Field>>

Other pointers are also supported, for a list, see here.

Motivation

Currently, there are some map functions that provide this functionality. These functions are not as ergonomic as a normal field access would be:

struct Count {
    inner: usize,
    outer: usize,
}
fn do_stuff(debug: Option<&mut Count>) {
    // something that will be tracked by inner
    if let Some(inner) = debug.map(|c| &mut c.inner) {
        *inner += 1;
    }
    // something that will be tracked by outer
    if let Some(outer) = debug.map(|c| &mut c.outer) {
        *inner += 1;
    }
}

With this RFC this would become:

struct Count {
    inner: usize,
    outer: usize,
}
fn do_stuff(debug: Option<&mut Count>) {
    // something that will be tracked by inner
    if let Some(inner) = &mut debug.inner {
        *inner += 1;
    }
    // something that will be tracked by outer
    if let Some(outer) = &mut debug.outer {
        *inner += 1;
    }
}

While this might only seem like a minor improvement for Option<T> it is transformative for Pin<P> and MaybeUninit<T>:

struct Count {
    inner: usize,
    outer: usize,
}
fn init_count(mut count: Box<MaybeUninit<Count>>) -> Box<Count> {
    let inner: &mut MaybeUninit<usize> = count.inner;
    inner.write(42);
    count.outer.write(63);
    unsafe {
        // SAFETY: all fields have been initialized
        count.assume_init() // #![feature(new_uninit)]
    }
}

Before, this had to be done with raw pointers!

Pin<P> has a similar story:

struct RaceFutures<F1, F2> {
    // Pin is somewhat special, it needs some way to specify
    // structurally pinned fields, because `Pin<&mut T>` might
    // not affect the whole of `T`.
    #[pin]
    fut1: F1,
    #[pin]
    fut2: F2,
}
impl<F1, F2> Future for RaceFutures<F1, F2>
where
    F1: Future,
    F2: Future<Output = F1::Output>,
{
    type Output = F1::Output;

    fn poll(self: Pin<&mut Self>, ctx: &mut Context) -> Poll<Self::Output> {
        match self.fut1.poll(ctx) {
            Poll::Pending => self.fut2.poll(ctx),
            rdy => rdy,
        }
    }
}

Without this proposal, one would have to use unsafe with Pin::map_unchecked_mut to project the inner fields.

Guide-level explanation

MaybeUninit<T>

When working with certain wrapper types in rust, you often want to access fields of the wrapped types. When interfacing with C one often has to deal with uninitialized data. In rust uninitialized data is represented by MaybeUninit<T>. In the following example we demonstrate how one can initialize partial fields using MaybeUninit<T>.

#[repr(C)]
pub struct MachineData {
    incident_count: u32,
    device_id: usize,
    device_specific: *const core::ffi::c_void,
}

extern "C" {
    // provided by the C code
    /// Initializes the `device_specific` pointer based on the value of `device_id`.
    /// Returns -1 on error (unknown id) and 0 on success.
    fn lookup_device_ptr(data: *mut MachineData) -> i32;
}

pub struct UnknownId;

impl MachineData {
    pub fn new(id: usize) -> Result<Self, UnknownId> {
        let mut this = MaybeUninit::<Self>::uninit();
        // the type of `this.device_id` is `MaybeUninit<usize>`
        this.device_id.write(id);
        this.incident_count.write(0);
        // SAFETY: ffi-call, `device_id` has been initialized
        if unsafe { lookup_device_ptr(this.as_mut_ptr()) } != 0 {
            Err(UnknownId)
        } else {
            // SAFETY: all fields have been initialized
            Ok(unsafe { this.assume_init() })
        }
    }
}

So to access a field of MaybeUninit<MachineData> we can use the already familiar syntax of accessing a field of MachineData/&MachineData /&mut MachineData. The difference is that the type of the expression this.device_id is now MaybeUninit<usize>.

These field projections are also available on other types.

Pin<P> projections

Our second example is going to focus on Pin<P>. This type is a little special, as it allows unwrapping while projecting, but only for specific fields. This information is expressed via the #[unpin] attribute on the given field.

struct RaceFutures<F1, F2> {
    fut1: F1,
    fut2: F2,
    // this will be used to fairly poll the futures
    #[unpin]
    first: bool,
}
impl<F1, F2> Future for RaceFutures<F1, F2>
where
    F1: Future,
    F2: Future<Output = F1::Output>,
{
    type Output = F1::Output;

    fn poll(self: Pin<&mut Self>, ctx: &mut Context) -> Poll<Self::Output> {
        // we can access self.first mutably, because it is `#[unpin]`
        self.first = !self.first;
        if self.first {
            // `self.fut1` has the type `Pin<&mut F1>` because `fut1` is a pinned field.
            // if it was not pinned, the type would be `&mut F1`.
            match self.fut1.poll(ctx) {
                Poll::Pending => self.fut2.poll(ctx),
                rdy => rdy,
            }
        } else {
            match self.fut2.poll(ctx) {
                Poll::Pending => self.fut1.poll(ctx),
                rdy => rdy,
            }
        }
    }
}

Defining your own wrapper type

First you need to decide what kind of projection your wrapper type needs:

  • field projection: this allows users to project &mut Wrapper<Struct> to &mut Wrapper<Field>, this is only available on types with #[repr(transparent)]
  • inner projection: this allows users to project Wrapper<&mut Struct> to Wrapper<&mut Field>, this is not available for unions

Field projection

Annotate your type with #[field_projecting($T)] where $T is the generic type parameter that you want to project.

#[repr(transparent)]
#[field_projecting(T)]
pub union MaybeUninit<T> {
    uninit: (),
    value: ManuallyDrop<T>,
}

Inner projection

Annotate your type with #[inner_projecting($T, $unwrap)] where

  • $T is the generic type parameter that you want to project.
  • $unwrap is an optional identifier, that - when specified - is available to users to allow projecting from Wrapper<Pointer<Struct>> -> Pointer<Field> on fields marked with #[$unwrap].
#[inner_projecting(T)]
pub enum Option<T> {
    Some(T),
    None,
}

Here is Pin as an example with $unwrap:

#[inner_projecting(T, unpin)]
pub struct Pin<P> {
    pointer: P,
}

// now the user can write:
struct RaceFutures<F1, F2> {
    fut1: F1,
    fut2: F2,
    #[unpin]
    first: bool,
}

&mut race_future.first has type &mut bool, because it is marked by #[unpin].

Reference-level explanation

Here is the list of types from core that will be field_projecting:

These will be inner_projecting:

Supported pointers

These are the pointer types that can be used as P in P<Wrapper<Struct>> -> P<Wrapper<Field>> for field_projecting and in Wrapper<P<Struct>> -> Wrapper<P<Field>> for inner_projecting:

  • &mut T, &T
  • *mut T, *const T, NonNull<T>, AtomicPtr<T>
  • Pin<P> where P is from above

Note that all of these pointers have the same size and all can be transmuted to and from *mut T. This is by design and other pointer types suggested should follow this. There could be an internal trait

trait NoMetadataPtr<T> {
    fn into_raw(self) -> *mut T;

    /// # Safety
    /// The supplied `ptr` must have its origin from either `Self::into_raw`, or
    /// be directly derived from it via field projection (`ptr::addr_of_mut!((*raw).field)`)
    unsafe fn from_raw(ptr: *mut T) -> Self;
}

that then could be used by the compiler to ease the field projection implementation.

Implementation

Add two new attributes: #[field_projecting($T)] and #[inner_projecting($T)] both taking a generic type parameter as an argument.

#[field_projecting($T)]

Restrictions

This attribute is only allowed on #[repr(transparent)] types where the only field has the layout of $T. Alternatively the type is a ZST.

How it works

This is done, because to do a projection, the compiler will mem::transmute::<&mut Wrapper<Struct>, *mut Struct> and then get the field using ptr::addr_of_mut! after which the pointer is then again mem::transmute::<*mut Field, &mut Wrapper<Field>>d to yield the projected field.

#[inner_projecting($T, $unwrap)]

Restrictions

This attribute cannot be added on unions, because it is unclear what field the projection would project. For example:

#[inner_projecting($T)]
pub union WeirdPair<T> {
    a: (ManuallyDrop<T>, u32),
    b: (u32, ManuallyDrop<T>),
}

$unwrap can only be specified on #[repr(transparent)], because otherwise Wrapper<Pointer<Struct>> cannot be projected to Pointer<Field>.

How it works

Each field mentioning $T will either need to be a ZST, or #[inner_projecting] or $T. The projection will work by projecting each field of type Pointer<$T> (remember, we are projecting from Wrapper<Pointer<Struct>> -> Wrapper<Pointer<Field>>) to Pointer<$F> and construct a Wrapper<Pointer<$F>> in place (because Pointer<$F> will have the same size as Pointer<$T> this will take up the same number of bytes, although the layout might be different). The last step will be skipped if the field is marked with #[$unwrap].

Interactions with other language features

Bindings

Bindings could also be supported:

struct Foo {
    a: usize,
    b: u64,
}

fn process(x: &Cell<Foo>, y: &Cell<Foo>) {
    let Foo { a: ax, b: bx } = x;
    let Foo { a: ay, b: by } = y;
    ax.swap(ay);
    bx.set(bx.get() + by.get());
}

This also enables support for enums:

enum FooBar {
    Foo(usize, usize),
    Bar(usize),
}

fn process(x: &Cell<FooBar>, y: &Cell<FooBar>) {
    use FooBar::*;
    match (x, y) {
        (Foo(a, b), Foo(c, d)) => {
            a.swap(c);
            b.set(b.get() + d.get());
        }
        (Bar(x), Bar(y)) => x.swap(y),
        (Foo(a, b), Bar(y)) => a.swap(y),
        (Bar(x), Foo(a, b)) => b.swap(x),
    }
}

They however seem not very compatible with MaybeUninit<T> (more work needed).

Pin projections

Because Pin<P> is a bit special, as it is the only Wrapper that permits access to raw fields when the user specifies so. It needs a mechanism to do so. This proposal has chosen an attribute named #[unpin] for this purpose. It would only be a marker attribute and provide no functionality by itself. It should be located either in the same module so ::core::pin::unpin or at the type itself ::core::pin::Pin::unpin.

There are several problems with choosing #[unpin] as the marker:

  • poor migration support for users of pin-project
  • not yet resolved the problem of PinnedDrop that can be implemented more easily with #[pin], see below.

Alternative: specify pinned fields instead (#[pin])

An additional challenge is that if a !Unpin field is marked #[pin], then one cannot implement the normal Drop trait, as it would give access to &mut self even if self is pinned. Before this did not pose a problem, because users would have to use unsafe to project !Unpin fields. But as this proposal makes this possible, we have to account for this.

The solution is similar to how pin-project solves this issue: Users are not allowed to implement Drop manually, but instead can implement PinnedDrop:

pub trait PinnedDrop {
    fn drop(self: Pin<&mut Self>);
}

similar to Drop::drop, PinnedDrop::drop would not be callable by normal code. The compiler would emit the following Drop stub for types that had #[pin]ned fields and a user specified PinnedDrop impl:

impl Drop for $ty {
    fn drop(&mut self) {
        // SAFETY: because `self` is being dropped, there exists no other reference
        // to it. Thus it will never move, if this function never moves it.
        let this = unsafe { ::core::pin::Pin::new_unchecked(self) };
        <Self as ::core::ops::PinnedDrop>::drop(this)
    }
}

To resolve before merge:

We could of course set an exception for pin and mark fields that keep the wrapper in contrast to other types. But Option does not support projecting "out of the wrapper" so this seems weird to make a general option.

Drawbacks

  • Users currently relying on crates that facilitate field projections (see prior art) will have to refactor their code.
  • Increased compiler complexity:
    • longer compile times
    • potential worse type inference

Rationale and alternatives

This RFC consciously chose the presented design, because it addresses the following core issues:

  • ergonomic field projection for a wide variety of types with user accesible ways of implementing it for their own types.
  • this feature integrates well with itself and other parts of the language.
  • the field access operator . is not imbued with additional meaning: it does not introduce overhead to use . on &mut MaybeUninit<T> compared to &mut T.

In particular this feature will not and should not in the future support projecting types that require additional maintenance like Arc. This would change the meaning of . allowing implicit creations of potentially as many Arcs as one writes ..

Out of scope: Arc projection

With the current design of Arc it is not possible to add field projection, because the refcount lives directly adjacent to the data. Instead the stdlib should include a new type of Arc (or ProjectedArc<Field, Struct>) that allows projection via a map function:

pub struct ProjectedArc<T, S> {
    backing: Arc<S>,
    ptr: NonNull<T>,
}

impl<T> Arc<T> {
    pub fn project<U>(&self, map: impl FnOnce(&T) -> &U) -> ProjectedArc<U, T> {
        ProjectedArc {
            backing: self.clone(),
            ptr: NonNull::from(map(&**self)),
        }
    }
}

What other designs have been considered and what is the rationale for not choosing them?

This proposal was initially only designed to enable projecting Pin<&mut T>, because that would remove the need for unsafe when pin projecting.

It seems beneficial to also provide this functionality for a wider range of types.

What is the impact of not doing this?

Users of these wrapper types need to rely on crates listed in prior art to provide sensible projections. Otherwise they can use the mapping functions provided by some of the wrapper types. These are however, rather unergonomic and wrappers like Pin<P> require unsafe.

Prior art

Crates

There are some crates that enable field projections via (proc-)macros:

  • pin-project provides pin projections via a proc macro on the type specifying the structurally pinned fields. At the projection-site the user calls a projection function .project() and then receives a type with each field replaced with the respective projected field.
  • field-project provides pin/uninit projection via a macro at the projection-site: the user writes proj!($var.$field) to project to $field. It works by internally using unsafe and thus cannot pin-project !Unpin fields, because that would be unsound due to the Drop impl a user could write.
  • cell-project provides cell projection via a macro at the projection-site: the user writes cell_project!($ty, $val.$field) where $ty is the type of $val. Internally, it uses unsafe to facilitate the projection.
  • pin-projections provides pin projections, it differs from pin-project by providing explicit projection functions for each field. It also can generate other types of getters for fields. pin-project seems like a more mature solution.
  • project-uninit provides uninit projections via macros at the projection-site uses unsafe internally.

All of these crates have in common that their users have to use macros when they want to perform a field projection.

Other languages

Other languages generally do not have this feature in the same extend. C++ has shared_ptr which allows the creation of another shared_ptr pointing at a field of a shared_ptr's pointee. This is possible, because shared_ptr is made up of two pointers, one pointing to the data and another pointing at the ref count. While this is not possible to add to Arc without introducing a new field, it could be possible to add another Arc pointer that allowed field projections. See this section for more, as this is out of this RFC's scope.

RFCs

Further discussion

Unresolved questions

Before merging

  • Is new syntax for the borrowing necessary (e.g. &pin mut x.y or &uninit mut x.y)?
  • how do we disambiguate field access when both the wrapper and the struct have the same named field? MaybeUninit<Struct>.value and Struct also has .value.

Before stabilization

  • How can we enable users to leverage field projection? Maybe there should exist a public trait that can be implemented to allow this.
  • Should unions also be supported?
  • How can enum and MaybeUninit<T> be made compatible?

Future possibilities

Arrays

Even more generalized projections e.g. slices: At the moment

exist, maybe there is room for generalization here as well.

Rc<T> and Arc<T> projections

While out of scope for this RFC, projections for Rc<T> and Arc<T> could be implemented by adding another field that points to the ref count. This RFC is designed for low cost projections, modifying an atomic ref count is too slow to let it happen without explicit opt-in by the programmer and as such it would be better to implement it via a dedicated map function.

There has been a discussion on zulip. It started as an effort to provide ergonomic pin-projections.

Here are some things that I want to improve before creating a pull request:

  • extend the guide-level explanation with:
    • enum
    • PinnedDrop

Changelist:

Edit #1
- added chapter on `Arc`/`Rc`

Edit #2
- Removed `Ref[Mut]`
- clarified Pin and pin example
- added attribute implementation details
- projecting arc section
- specified projectable pointers
- elaborated on design rationale
- updated unresolved questions

Edit #3
- fixed task lists
- added field name disambiguation question
15 Likes
2

It would be extremely nice if we could get field projections for Rc and Arc as well. Those tend to be stored and passed around as owned objects, so any field reference must necessarily be either short-lived or be represented separately in the reference count. At the moment, only the first option is available without significant effort.

They’re technically more challenging, so may be out of scope for this RFC, but they probably deserve a mention in the alternatives or future work sections,

3

Thanks for the suggestion, I added this to the future work section:

Rc<T> and Arc<T> projections

While out of scope for this RFC, projections for Rc<T> and Arc<T> could be implemented in a similar way. This change seems to be a lot more involved and will probably require that more information is stored in these pointers. It seems more likely that this could be implemented for a new type that explicitly opts in to provide field projections.

4

Another possible thing that could be interesting here: repr(packed) causes a bunch of problems because the field types lie, in a sense. But if we used Packed<Foo> instead, such that one could get a &Packed<i32> from field access, a bunch of those problems go away.

This makes two things jump out at me:

  1. This is different from how field access works on references. If foo.bar turns a Pin<Foo> into a Pin<Bar>, then that seems inconsistent with how foo.bar turns a &Foo into a place Bar, not a &Bar. Is that ok? Is it an indication that the way . works for references is "wrong" somehow?

  2. What are the rules for name conflicts? For example, MaybeUninit is a union with a value field. So what happens if I do this.value? Am I blocked from using projection if my type has a field of that name? How do I disambiguate if I'm in core and have vis access to both kinds of field? (This reminds me of Deref-style naming conflicts, which are why MaybeUninit and ManuallyDrop have such different APIs.)

Basically this would be a type that works more like C++'s shared_ptr, which has an aliasing constructor -- you can put any value into it, and it keeps the underlying storage alive.

That's not free, though, since you can no longer trust that the refcount for a shared_ptr<T> going to zero will deallocate a T, so it needs to store what C++ calls a deleter that knows what dropping-and-deallocation logic is needed.

3 Likes
5

That seems very nice!

For Pin<P> to exist, P needs to be Deref, also I think you meant if foo: Pin<&mut Foo> then foo.bar: Pin<&mut Bar>. References seem to be inconsistent with this behavior.

That is a very good point! I think we will have to create some new syntax for the field access of wrappers, at the moment this would only be needed internally.

That is why I suggested creating a new type, rather than extending Arc.

6

You would also want to support field projections for raw pointers *const T/*mut T. More importantly, we should be able to field-project *(const/mut) MaybeUninit<T> to *(const/mut) MaybeUninit<Field>, and similarly for UnsafeCell, because in unsafe code you would often deal with pointers, and would want to avoid giving at any moment any pointer validity/aliasing guarantees.

This isn't currently possible to do in end-user code without transmutes and dealing with offsets of fields. There is currently an RFC for offset_of! macro, which would give you field offsets, including #[repr(Rust)] types. Personally I'm not happy with that addition. It would be nice if there was some way to get the benefits of field offsets without directly exposing them to the users.

Some way to implement field projections is highly desirable, but I wouldn't want it to be a compiler built-in. There are way too many possible types to implement this feature in an ad-hoc way, it's not just a matter of a couple new lang items. This functionality is also highly desirable for user smart pointers.

Adding new methods to those types is relatively easy, but that would be highly unsafe, since there is no way to verify the offsets of fields (or pointers to fields) provided to the methods.

Derive macros would also be a neat solution, but that would mean that projections would work only if the author of the inner type has bothered to provide all required derives. Again, it seems infeasible to add 20 derive macros to each struct, and a catch-all derive macro would not provide the level of control generally expected of libcore derive macros. There is also an issue that some of the types live in core, while some require alloc or std. That would incur the same fragmentation for the derive macros. Finally, the macro-based approach isn't composable: you have derived Option<&T> and &MaybeUninit<T>, but have you derived Option<&MaybeUninit<T>>? This sounds like a frustrating and unexpected limitation.

I would expect so. You can map *mut Union to *mut Variant via offset_of!, why wouldn't it be possible for smart pointers?

Crazy thought: what if we had first-class place expressions in the language?

To avoid derailing this thread with pie-in-the-sky proposals, I have opened a separate topic.

5 Likes
7

You can use addr_of[_mut]!() for that, so the status quo is not that bad in this area.

1 Like
8

As @chrefr already mentioned, I also think that for *const/*mut T this is already rather ergonomic especially when we get &raw. But for *const UnsafeCell<T> I agree with you [1].

I would like to see some more examples (maybe also out in the wild, if you have some crates in mind that would benefit, please let me know!). I found it very challenging to find a good, safe API that could support all use-cases. &MaybeUninit<X> -> &MaybeUninit<Y> forces us to use raw pointers, because of uninitialized memory [2].

I think it is a mistake to allow field projections for core::cell::Ref/core::cell::RefMut, it already implements Deref/DerefMut and nothing is really gained, because one could access the fields anyway. APIs should not be designed to take a Ref[Mut] directly, so this should be a non-issue.

Removing Ref[Mut] would allow us to take a different approach to field projection:

pub trait FieldProject<'a, T> {
    fn as_raw(self) -> *mut T;
    unsafe fn from_raw(ptr: *mut T) -> Self;
}
// e.g.
impl<'a, T> FieldProject<'a, T> for &'a mut MaybeUninit<T> {
    fn as_raw(self) -> *mut T { self }
    unsafe fn from_raw(ptr: *mut T) -> Self {
        unsafe { &mut *ptr }
    }
}

This form of API would prevent any implementations that would have to track more state (e.g. increment a reference count/copy borrow information like Ref[Mut]). But I feel more at ease if this were public compared to the current proposal.

I do not quite understand what you mean by this, could you elaborate?

This cannot be done in safe code, I wanted to create safe and ergonomic ways of projection. One could already unsafely project to the fields of &mut MaybeUninit<T>, but to me this unsafety seemed entirely unnecessary.

  1. I would actually really like to see *const/*mut Self as a receiver type, such that raw_get could be directly called on *mut UnsafeCell<T>, but projecting into fields here also seems like a hassle. ↩︎

  2. We also need them for Cell<T>/ UnsafeCell<T>. ↩︎

9

Sounds like that would also rule out the previously mentioned use case of supporting projections for reference-counted types.

Prior art: C++'s equivalent of Arc, std::shared_ptr, actually consists of two pointers. One pointer is used when dereferencing the shared_ptr, while the other points to the reference count.

Normally, this is redundant and a waste of memory. Rust Arc only consists of a single pointer: the reference count is at a fixed offset from the object being reference-counted, so Arc only needs one pointer to access both of them.

However, C++'s approach lets you have a shared_ptr that uses one object's reference count while dereferencing to another object entirely. For example, you can 'project' a shared_ptr to a struct into a shared_ptr to one of its fields. The projected shared_ptr keeps the entire struct alive (as it must, since the field is not its own heap allocation), but dereferences to the field. Naturally, it doesn't have to be a field; it could be any memory owned directly or indirectly by the original object, like, say, a single element of a std::vector.

(That's not the only reason that C++'s shared_ptr uses two pointers, but the other reasons aren't applicable to Rust.)

Rust definitely made the right choice in making Arc a single pointer. But it could benefit from having a separate type which is two pointers and supports field projection.

Admittedly, Arc field projection doesn't need to use built-in field projection syntax. In C++ it's just a (completely unsafe) API. In Rust it could be done as an API using a map function, similar to the ones on Ref and RefMut. Indeed, yoke and owning_ref both do something like this and could be used for projected Arcs, although both crates are a bit more ambitious and flexible than what I'm imagining.

Still, if we're going to have field projection syntax, it feels odd to not support this.

5 Likes
10

Thanks for sharing! I will add that

An additional argument in favor of explicit Arc projection would be that it is not as cheap as other conversions. The field projection of &mut MaybeUninit<Struct> -> &mut MaybeUninit<Field> is exactly as costly as &mut Struct -> &mut Field. But when doing the same thing with an Arc, then the reference count would have to be touched. I do not think this would be good design.

For me, it is enough to say field projection syntax is only going to do pointer offset and zero cost abstraction stuff.

I also do not really see the use cases for allowing field projections on Arc. I believe one could always just use &Field or propagate the whole Arc instead (I have not tried to find an example where it would be very useful, I just think that one could use alternatives relatively easily and it would not really hurt.).

2 Likes
11

The motivating use case for me goes something like this: Implement an Iterator that is 'static and yields some kind of reference. It will need to hold the container in an Arc and yield projections of the container items.

For more advanced cases, you may want the iterator to hold a MutexGuard which in turn keeps an Arc<Mutex<…>> alive via projection.

Edit: On reflection, this second case is probably a cleaner argument. If you have an Arc<Mutex<…>>, there’s no current way to hold the lock with a 'static guard object. Instead, you need to come up with a lifetime that is guaranteed to live longer than the lock needs to be held and convert the Arc into an ordinary reference at that point in the call stack.

12

Could you give a concrete example? What would the lifetime of the reference be that is returned?

I do not understand, how does field projection play a role in this?

13

Something like this:

struct ArcIter<T> {
    items: Arc<[T]>,
    idx: usize 
}

impl<T> Iterator for ArcIter<T> {
    type Item = ProjectedArc<T>;
    fn next(&mut self)->Option<Self::Item> {
        let result = if self.idx < self.items.len() {
            Some(Arc::project(&self.items, |v| &v[self.idx]))
        } else { None }
        self.idx += 1;
        result
    }
}

Edit: Made theoretical code more plausible

3 Likes
14

To be clear I am not against Arc projection. I am just against allowing arc.field to resolve to ProjectedArc<Field>, because it is different from the other projections.

15

I don't have a particular preference as to how Arc projection gets implemented, but you asked (by implication) for concrete examples of how it might be useful, so I provided one: You can't propagate the entire Arc because you need something that's Borrow<T>, and you can't just take a reference because iterators must yield items that can outlive themselves.

You are correct here; this problem would be solved by making MutexGuard generic over Borrow<Mutex<...>>, which feels related but would be a completely separate proposal. My apologies for not realizing this was a separate issue. You might then want to use field projection on that resulting guard, though.

Aside: I just noticed that you're providing projection for RefCell's guards, but not Mutex's or RwLock's. It feels odd to leave out these Sync types that otherwise do the same job.

16

Yes thank you for your example, earlier I was still on the fence if it is a good idea to allow Arc projection. Now I believe that because of the need to change the atomic ref count that we should make it explicit and only allow pointer offset style projections to actually have the field projection syntax struct.field.

That is correct, but I was arguing to remove RefCells guards, because they pose the same "metadata modification" issue as Arc. So I think MutexGuard would also fall under this category.

17

This will also rule out projections over Option.

18

Oh I did not think about that...

I also do not like the interface that would provide for the users. What if we would provide two new attributes on types:

#[field_projecting]

This attribute is only available on types with #[repr(transparent)]

#[repr(transparent)]
#[field_projecting]
pub union MaybeUninit<T> {
    uninit: (),
    value: ManuallyDrop<T>,
}

This will result in allowing transformations of Pointer<MU<Struct>> -> Pointer<MU<Field>> via struct.field. Where Pointer is some pointer type.

This attribute will be sufficient for honest wrapper types that only give type information but do not change the memory layout:

  • MaybeUninit<T>
  • Cell<T>
  • UnsafeCell<T>

#[inner_projecting]

This attribute is available on all Sized struct and enum types (unions are not supported, as the active field is not known at runtime) with exactly one generic type parameter. And where every field mentioning T is projectable.

#[inner_projecting]
pub enum Option<T> {
    Some(T),
    None
}

This will result in allowing transformations of Option<Pointer<Struct>> -> Option<Pointer<Field>> . Each field would be projected and fields without T would be kept as-is (because the pointer size is the same, this could even be done in place).

Combination

These two attributes would be made compatible, such that Option<&mut MU<T>> would also be projectable. More complex type: Pin<&mut MaybeUninit<UnsafeCell<T>>>.

This approach would easily allow users to specify their own wrapper types without having to define a complex unsafe trait impl.

19

What’s the motivation for restricting this to single-type generics? Is there some problem with allowing projection into multiple different types depending on the field name?

As a concrete example, why should this be disallowed?

#[inner_projecting]
pub struct Pair<K,V> {
    pub key: K,
    pub value: V
}

Maybe the attribute should go on the projectable fields instead:

pub struct Pair<K,V> {
    #[allow_projection] pub key: K,
    #[allow_projection] pub value: V
}
20

How would you disambiguate if K and V have the same field? Would it project both? Can you choose (what syntax?)? For the time being I chose to keep it simple.

I am not sure, I think it should go on the generic parameter, because what about this:

pub struct Pair<K,V> {
    #[allow_projection] pub inner: (K, V),
}

Sidenote: should tuples/slices/arrays/other primitives be #[inner_projecting]?