[Pre-RFC] Reborrow trait

[aapoalas]

Well hello there!

I'm soliciting for final feedback on the third draft of a Reborrow / "autoreborrow traits" RFC that I've been drawing up over the previous months. The RFC is up in my repository and the most recent version is included below. If you have the time to read it, I'd welcome any and all comments you have!

---

• Feature Name: autoreborrow-traits
• Start Date: 2025-04-15
• RFC PR: rust-lang/rfcs#0000)
• Rust Issue: rust-lang/rust#0000)

Summary

Enable users to opt into automatic reborrowing of custom types with exclusive reference semantics.

Motivation

Reborrowing of exclusive references is an important feature in Rust, enabling much of the borrow checker's work. This important feature is not available for reference wrappers or user-defined reference-like types, which hampers the usefulness of the language when working with these kinds of types. Manual reborrow-like functions can be implemented but they must be explicitly called making them verbose, and are importantly always limited to creating reference-like types bound to the local function that cannot be returned to the caller.

Examples of features unlocked by this RFC are:

• Automatic reborrowing of Option<&mut T> and Pin<&mut T>.
• Automatic reborrowing of custom exclusive reference types like Mut<'_, T>.
• Returning of values deriving from reborrowed parameters, extending to the source lifetime.

A reborrow is a bitwise copy of a type with extra lifetime analysis performed by the compiler. The lifetime of the reborrow result is a subset of the source value's lifetime (not a strict subset, the result's lifetime is allowed to be equal to the source's lifetime). When reborrowing as exclusive, the source value is disabled for the lifetime of the result. When reborrowing as shared, the source value is disabled for mutations for the lifetime of the result.

Status quo - reference reborrowing

This section gives basic Rust reference usage examples with reborrowing. This is to serve as a comparison for later examples of autoreborrow traits usage. The following kind of methods are assumed to exist on all reference-like types in these examples.

impl<T, U> T {
    /// Requires exclusive access to self and returns some derived exclusive reference.
    fn mutate(self) -> U;
    /// Requires shared access to self and returns some derived shared reference.
    fn observe(self) -> U;
}

Note that the methods are written as taking self by value; because we're talking about reference-like types, this is intended to be read as fn mutate(&mut self) and fn observe(&self) for normal references and eg. fn mutate(self: MyMut) and fn observe(self: MyRef) for custom reference types.

The following examples of reborrowing will be used later:

fn mixed_reborrowing(source: &mut T) {
    // Exclusive reborrow disables source.
    {
        let result: &mut U = source.mutate();
        // The following line would not compile because source is disabled for
        // the duration of the result's lifetime:
        // source.observe();

        result.observe();
    }
    // Note: source can be reborrowed as shared or exclusive after the
    // previous exclusive reborrow is dropped.

    // Shared reborrow disables source for mutations. Source can be reborrowed
    // as shared multiple times.
    {
        let result1: &U = source.observe();
        let result2: &U = source.observe();
        // The following line would not compile because source is disabled
        // for mutations for the duration of the results' lifetime:
        // source.mutate();

        // Note: both shared reborrow results can be kept and used
        // simultaneously.
        result1.observe();
        result2.observe();
    }
    // Note: after all shared reborrow results are dropped, source can be
    // reborrowed as exclusive again.
    source.mutate();
}

fn returning_derived_mut(source: &mut T) -> &mut U {
    // Note: the reborrow that happens when calling mutate() extends its
    // lifetime to the lifetime of source, not to some local value conceptually
    // create here:
    // let reborrow = &mut *source;
    // reborrow.mutate()
    source.mutate()
}

fn returning_derived_ref(source: &mut T) -> &U {
    // Note: exclusive reference is reborrowed as shared for the return value.
    source.mutate()
}

// Some function returning a result where the both variants refer to the
// source data; think eg. a parser that wants to return a structure containing
// references to source text slice or an error pointing to invalid source text.
fn result_function(source: &T) -> Result<&X, &Y>;

// A function taking an exclusive reference, reborrowing it as shared to call
// other functions that return derived references, and then conditionally
// returning some results of those calls.
fn branching_function(source: &mut T) -> Result<&U, &Y> {
    // Note: the early Err return requires the Err value's lifetime to extend
    // to the source lifetime, ie. to the end of the call. The mutate call
    // later requires the Ok value's lifetime to end before that call. The
    // compiler is currently not smart enough to split Ok and Err from one
    // another, meaning that this does not work in current stable Rust.
    // However, this does compile on Polonius!
    let result = result_function(source)?;
    println!("Example: {:?}", result);
    source.mutate()
}

The important things to note here are:

1. Reborrowing is done on exclusive references only; shared references are Copy and need no reborrowing semantics.
2. Reborrowing has two flavours, shared and exclusive.
3. Reborrowing happens implicitly at each coercion-site.
4. If the function being called returns a value derived from the reborrowed reference, the returned value's lifetime extends in the caller's scope to match the original reference's lifetime.
5. Branching function returns run into issues with lifetime extension that needs Polonius to resolve.

Wrapper type reborrowing

Wrapper types like Option<T> cannot be reborrowed today. This makes using optional exclusive references tedious as they are automatically moved when used. Reborrowing of Pin<&mut T> has been provisionally added into the nightly compiler, but it is done via custom handling and relies on internal knowledge of the type.

We'll revisit the earlier examples, now with Option<&mut T> as our example of a wrapper type with exclusive reference semantics. Note that this means that the function signatures for our helper functions are now fn mutate(self: Option<&mut T>) -> Option<&mut U> and fn observe(self: Option<&T>) -> Option<&U>.

fn mixed_reborrowing(source: Option<&mut T>) {
    // Exclusive reborrowing
    {
        // Note: In current Rust this would work, but after this the source has
        // been moved out of this function and could no longer be used. This
        // call site must change from using a move to using a reborrow.
        let result: Option<&mut U> = source.mutate();
        // Again, we want the following line to fail to compile, and it does
        // but currently for the wrong reasons:
        // source.mutate();

        result.observe();
    }

    // Shared reborrowing
    {
        // We want to enable the following calls to still work: this means that
        // the first mutate() call must reborrow source instead of moving it.
        // The following calls must also implicitly reborrow source as shared.
        let result1: Option<&U> = source.observe();
        let result2: Option<&U> = source.observe();
        result1.observe();
        result2.observe();
    }
    // After all shared reborrow results are dropped, source must be
    // reborrowable as exclusive again.
    source.mutate();
}

fn returning_derived_mut(source: Option<&mut T>) -> Option<&mut U> {
    // This does work today, but mutate() must internally map() the Option
    // instead of eg. using as_deref_mut() to compile. See below.
    source.mutate()
}

fn returning_derived_ref(source: Option<&mut T>) -> Option<&U> {
    // Note: This does not work today, and suggests using as_deref() which also
    // doesn't work, citing "returns a value referencing data owned by the
    // current function". Using eg. Option::map() to inject a coercion site
    // fixes the issue:
    // source.map(|v| -> &T { v })
    source.mutate()
}

// Some function returning a result where the both variants refer to the
// source data; think eg. a parser that wants to return a structure containing
// references to source text slice or an error pointing to invalid source text.
fn result_function(source: Option<&T>) -> Result<&X, &Y>;

// A function taking an exclusive reference, reborrowing it as shared to call
// other functions that return derived references, and then conditionally
// returning some results of those calls.
fn branching_function(source: Option<&mut T>) -> Result<&U, &Y> {
    // Note: this would again move source and hence does not work.
    // let result = result_function(source)?;
    // What about manually reborrowing? No, this does not work either!
    // The Option<&T> now has a lifetime bound on the temporary as_ref()
    // borrow and cannot be returned from this function (via the Err variant)
    // as it "returns a value referencing data owned by the current function".
    let result = result_function(source.as_ref().map(|v| -> &T { v }))?;
    // Do something with the result.
    println!("Example: {:?}", result);
    // Then mutate the source again.
    Ok(source.mutate())
}

The important thing to note here is that while manual unwrapping and rewrapping of an Option does help through many issues, it too fails in the branching function returns case, though it gives a different error: the error is now related to the lack of true reborrowing as opposed to the compiler not knowing how to split Ok and Err branch lifetimes from one another. Even Polonius does not help in this case, as the source.as_ref() call has made the result lifetime a strict subset of the source lifetime: even though we can see that the transformation from Option<&mut T> to Option<&T> does not retain any references to local values, the compiler cannot be quite so sure and thus does not allow this code to compile.

Custom reference type reborrowing

Custom reference types are as many as there are users, but a common theme amongst them is that they usually hold one (though sometimes multiple or none) data field and a PhantomData field (or multiple) for holding a lifetime in. The previous examples are once again revisited with custom MyMut and MyRef reference types. The function signatures of our helper functions are now fn mutate<'a>(self: MyMut<'a>) -> MyMut and fn observe<'a>(self: MyRef<'a>) -> MyRef.

In the examples, "FIX" and "NOT" comments are given to show how some of these examples can be made to work today and what sort of issues are related to those fixes. These fixes use rb() and rb_mut() helper methods that are "reborrow-like" but do not perform true reborrowing, as will become apparent in the examples.

/// Exclusive reborrowable reference to some data. Can be turned into a MyRef
/// by reborrowing as shared.
///
/// Note: this type is not Copy or Clone.
struct MyMut<'a>(...);
impl Reborrow for MyMut { /* ... */ }

/// Shared reference to some data.
#[derive(Clone, Copy)]
struct MyRef<'a>(...);

/// Reborrow-like helper methods, in the form of a trait.
trait Reborrow {
    /// Type that represents the Self exclusive reference reborrowed as shared.
    type Target;

    fn rb_mut(&mut self) -> Self;
    fn rb(&self) -> Self::Target;
}

fn mixed_reborrowing(source: MyMut) {
    // Exclusive reborrowing
    {
        // Note: mutate() takes self by value, so in current Rust MyMut would be
        // moved here. Instead we want it to be reborrowed as exclusive.
        // Note: explicit "reborrow-like" function works today:
        // FIX: let result = source.rb_mut().mutate();
        let result = source.mutate();
        // Again, we want the following line to fail to compile, and it does
        // but currently for the wrong reasons:
        // source.mutate();

        result.observe();
    }

    // Shared reborrowing
    {
        // We want to enable the following calls to still work: this means that
        // the first mutate() call must reborrow source instead of moving it.
        // The following calls must also implicitly reborrow source as shared.
        // Note: the explicit "reborrow-like" function works today:
        // FIX: let result1 = source.rb().observe();
        // FIX: let result2 = source.rb().observe();
        let result1 = source.observe();
        let result2 = source.observe();
        // Note: source can be reborrowed as shared multiple times.
        result1.observe();
        result2.observe();
    }
    // After all shared reborrow results are dropped, source must be
    // reborrowable as exclusive again.
    // Note: even in current Rust we don't need to use the "reborrow-like"
    // function here, as source is not used after this and we can thus move it
    // out of the function.
    source.mutate();
}

fn returning_derived_mut(source: MyMut) -> MyMut {
    // This does work today, but only if source is moved into mutate() instead
    // of using the "reborrow-like" function:
    // NOT: source.rb_mut().mutate()
    source.mutate()
}

fn returning_derived_ref(source: MyMut) -> MyRef {
    // This does not work today, but can be made to work with a helper, such as
    // the Into trait, that turns an owned MyMut into an owned MyRef. Again,
    // using the explicit "reborrow-like" function does not work:
    // FIX: source.mutate().into()
    // NOT: source.rb_mut().mutate().into()
    source.mutate()
}

// Some function returning an error where both variants refer to the source
// data; think eg. a parser that wants to return a structure containing
// references to source text slice or an error pointing to invalid source text.
fn result_function(source: MyRef) -> Result<MyRef, MyRef>;

// A function taking an exclusive reference, reborrowing it as shared to call
// other functions that return derived references, and then conditionally
// returning some results of those calls.
fn branching_function(source: MyMut) -> Result<MyRef, MyRef> {
    // Note: here again we'd want automatic reborrowing as shared. If we use
    // the explicit "reborrow-like" function then the resulting MyRef has a
    // lifetime strictly bound to the temporary borrow of source. That is what
    // we want for the Ok value, as we must drop result before source gets
    // reused as exclusive, but for the Err value we conversely want the
    // lifetime to be extended to the source lifetime so that we can return it
    // from the function. This could work with Polonius, but only with true
    // reborrowing instead of the "reborrow-like" functions. In effect, this
    // function cannot currently be compiled without using lifetime transmutes.
    // NOT: result_function(source.rb())?;
    // This also does not work, since source is moved here but reused below.
    let result = result_function(source)?;
    // Do something with the result.
    println!("Example: {:?}", result);
    // Then mutate the source again.
    Ok(source.mutate())
}

The most important point here is that just like with Option<&mut T>, custom exclusive reference-like types cannot be used in functions returning values with derived lifetimes from branches. The derived lifetime is long enough (equal to the source lifetime) if the reference-like type is moved out of the function and into the subroutine, but then it cannot be reused. If reference-like type is explicitly "reborrowed" using the helper functions, then the resulting derived lifetime is strictly bound within the local function and cannot be returned from it.

Let's look at usage of reborrowing, or the "reborrow-like" functions, of custom reference-like types in the wild. The following are a smattering of custom reference-like types and traits that either make use of "reborrow-like" functions, implement the "reborrow-like" function APIs, or are interested in using reborrowing in the future.

PeripheralRef

PeripheralRef is an custom exclusive reference-like type that provides exclusive access to hardware peripherals in an embedded system codebase. Of note is that the size is usually either a ZST or a single byte in size as many peripherals are either statically known and need no runtime behaviour, or have only a small number of possible values to refer to.

Also note that PeripheralRef does not have a corresponding shared reference-like type.

• Size: Generic type dependent, but commonly ZST or 1 byte.
• Kind: Zero-sized marker type, or a small value for runtime decision making.
• Usage: Provide exclusive access to statically knowable (or mostly knowable) hardware peripherals in an embedded system codebase.

GcScope and NoGcScope

GcScope and NoGcScope are a custom exclusive and shared reference-like type, respectively, that provide access to the Nova JavaScript engine's garbage collector. The types are zero-sized and have no runtime existence, serving only as a means of compile-time verification of garbage collector safety, or use-after-free checking in other words. The "garbage collector lifetime" of the type is observed by all garbage-collectable data handles, and is taken as exclusive by any method that may trigger garbage collection. The result is that methods that may trigger garbage collection always invalidate all garbage-colletable data handles on call. Of note is that these types actually carry a second lifetime as well, the "scope lifetime", which is used when handles are rooted to the current scope. This second lifetime is always shared reference-like, and thus does not participate in reborrowing.

• Size: ZST.
• Kind: Marker types with no runtime existence. GcScope has two lifetimes, one lifetime is exclusive and one is shared. NoGcScope has two shared lifetimes. The always-shared lifetime takes no part in reborrowing and can be ignored here.
• Usage: GcScope provides exclusive access to a garbage collector, while its shared reborrow NoGcScope is used to create handles to garbage collected data. Handles are thus invalidated if GcScope is used as exclusive.

MutableHandle and Handle

MutableHandle and Handle are custom reference-like types to garbage collectable data in the Servo browser's SpiderMonkey mozjs wrapper.

• Size: Pointer-sized.
• Kind: MutableHandle is conceptually an exclusive reference type. Handle is a wrapper around a shared reference.
• Usage: MutableHandle is a reference type wrapping a raw pointer that can unsafely be turned into an exclusive reference on the condition that the garbage collector does not run while the exclusive reference exists. Handle is a plain shared reference wrapper; the pointed-to value is (aspirationally) only mutated by the GC where it is internally mutable.

Mat, MatMut, MatRef, ColMut, and ColRef

Mat is an owned rectangular matrix while MatMut, MatRef, ColMut, and ColRef are various kinds of custom reference types over the source Mat.

• Size: Usually 5 pointer-sizes for MatMut and MatRef, 3 for ColMut and ColRef.
• Kind: MatMut and ColMut are exclusive reference types. MatRef and ColRef are shared reference types.
• Usage: Enable various near-zero-cost operations over matrices by binding the iteration to the original Mat through the reference types while statically ensuring that data access within the bounds of the borrow is valid.

Note: Mat is also reborrowable into MatMut and MatRef, but also has a Drop implementation. If reborrowing is understood to act on lifetimes of reference-like types, then reborrowing Mat is outside that definition as it neither has a lifetime nor is a reference-like type. It could be argued that Mat does have an implied 'static lifetime, but the lack of reference-like aspects of it at least cannot be denied. This use-case might be better thought of as a generalised AsRef/AsMut or perhaps Deref[Mut] implementation.

reborrow crate

This is a library used in eg. faer (above Mat and friends) that provides traits for emulating reborrowing.

Honorable mention: Rust for Linux smart pointers

The Rust for Linux project has many custom smart pointer types with peculiar requirements beyond normal Rust usage. At least one type implements reborrowing, and according to some discussion on Zulip, there is some interest in having more of such types. One of the reasons why more of such types do not exist is said to be bad ergnomics around automatic reborrowing.

Honorable mention: CppRef and friends

These types do not implement reborrowing and do not always carry lifetimes, but cxx is interested in automatic coercion between the different C++ reference types. Since these are pointer wrapping types, some forms of autoreborrowing would make these coercions possible.

Guide-level explanation

When creating a custom exclusive reference types, users can derive a Reborrow trait on the type which opts the type in to &mut like semantics. The trait checks that all of the fields of the type implement Reborrow and return false for needs_drop. The trait has no methods and no parameter or associated types. A blanket implementation of Reborrow for Copy types is provided by the core library.

For coercing a custom exclusive reference type into a shared reference type, users can implement a CoerceShared: Reborrow trait on the custom exclusive reference type. The trait is simple and only requires specifying the Target type:

trait CoerceShared: Reborrow {
    type Target: Copy;
}

The Copy bound on Target ensures that all shared reference types are always trivially copyable. The trait checks that for all fields in T::Target, a field with the same name exists in T such that type Field of the field in T implements is Field: CoerceShared where the type SharedField of the field in T::Target is equal to the type <Field as CoerceShared>::Target. A blanket implementation of CoerceShared<Target = Self> for Copy types is provided by the core library.

When a value T: Reborrow is used at a coercion site expecting T, a new T is created from the original value using the functional record update syntax and the original value is disabled for reads and writes. The original value stays disabled for writes until all fields of T that are Reborrow but not Copy are dropped, and stays disabled for reads until all fields of T that are Copy and carry a lifetime are dropped. This is considered a write on the original value.

When a value T: CoerceShared<Target = U> is used at a coercion site expecting U, a new U is created from the original value with each field in U performing an implied CoerceShared method on a copy of the corresponding field in T. The original value stays disabled for writes until all fields of U that are Copy and carry a lifetime are dropped. This is considered a read on the original value.

Custom exclusive to shared reference type method resolution

Implementing T: CoerceShared<Target = U> makes it possible to call methods of U on T. For example, for a type MyMut<'a, T>: Reborrow + CoerceShared<Target = MyRef<'a, T>> with a method

impl<'a, T> MyRef<'a, T> {
    fn get(self) -> &'a T;
}

it is possible to call this directly on a value of type T:

fn example<T>(t: MyMut<T>) {
    let data: &T = t.get();
}

This implicitly applies CoerceShared on MyMut<'a, T> to create a MyRef<'a, T> and finds the get method on that type.

Building custom reference types

A basic custom exclusive reference type is as follows:

struct MyMut<'a, T> {
    ptr: NonNull<T>,
    lifetime: PhantomData<&'a ()>,
}

The MyMut struct does not derive Copy and thus requires a Reborrow implementation. When MyMut<T> is reborrowed, the compiler translates

fn inner_mut(source: MyMut<T>);

fn outer_mut(source: MyMut<T>) {
    inner_mut(source);
}

into something akin to

fn inner_mut(source: MyMut<T>);

fn outer_mut(source: MyMut<T>) {
    let MyMut {
        ptr,
        lifetime,
    } = source;
    inner_mut(MyMut {
        ptr,
        lifetime,
    })
}

where the original source is marked used as exclusive by the compiler for the lifetime of the resulting MyMut, similarly to how it would work if lifetime was a &mut reference.

The corresponding custom shared reference type for MyMut is as follows:

#[derive(Clone, Copy)]
struct MyRef<'a, T> {
    ptr: NonNull<T>,
    lifetime: PhantomData<&'a ()>,
}

and using CoerceShared to convert a MyMut into a MyRef is translated into something akin to

fn inner_ref(source: MyRef<T>);

fn outer_ref(source: MyMut<T>) {
    let MyMut {
        ptr,
        lifetime,
    } = source;
    inner_ref(MyRef {
        ptr,
        lifetime,
    })
}

where the original source is marked used as shared for the lifetime of the MyRef.

Automatic reborrowing of structs with multiple fields

The field-wise definition of Reborrow and CoerceShared traits enables automatic reborrowing of structs that have multiple fields and even multiple lifetimes with different reference semantics associated with them.

A basic example, simplified from faer's reborrowing use case, with a single lifetime and multiple data fields is given below.

struct MatMut<'a, T> {
    ptr: NonNull<T>,
    rows: usize,
    cols: usize,
    row_stride: usize,
    col_stride: usize,
    __marker: PhantomData<&'a ()>,
}

#[derive(Clone, Copy)]
struct MatRef<'a, T> {
    ptr: NonNull<T>,
    rows: usize,
    cols: usize,
    row_stride: usize,
    col_stride: usize,
    __marker: PhantomData<&'a ()>,
}

fn inner_mut(source: MatMut<T>);

fn outer_mut(source: MatMut<T>) {
    // inner_mut(source); is turned by the compiler into:
    let MatMut {
        ptr,
        rows,
        cols,
        row_stride,
        col_stride,
        __marker,
    } = source;
    inner_mut(MatMut {
        ptr,
        rows,
        cols,
        row_stride,
        col_stride,
        __marker,
    })
}

fn inner_ref(source: MatRef<T>);

fn outer_ref(source: MatMut<T>) {
    // inner_ref(source); is turned by the compiler into:
    let MatMut {
        ptr,
        rows,
        cols,
        row_stride,
        col_stride,
        __marker,
    } = source;
    inner_ref(MatRef {
        ptr,
        rows,
        cols,
        row_stride,
        col_stride,
        __marker,
    })
}

It is also possible for the CoerceShared::Target type to have a different layout than the source type does. An example described by a user in their closed source code base is given below.

struct ImbrisMut<'a, T> {
    ptr: NonNull<T>,
    metadata: usize,
    __marker: PhantomData<&'a ()>,
}

#[derive(Clone, Copy)]
struct ImbrisRef<'a, T> {
    ptr: NonNull<T>,
    __marker: PhantomData<&'a ()>,
}

fn inner_mut(source: ImbrisMut<T>);

fn outer_mut(source: ImbrisMut<T>) {
    // inner_mut(source); is turned by the compiler into:
    let ImbrisMut {
        ptr,
        metadata,
        __marker,
    } = source;
    inner_mut(ImbrisMut {
        ptr,
        metadata,
        __marker,
    })
}

fn inner_ref(source: ImbrisRef<T>);

fn outer_ref(source: ImbrisMut<T>) {
    // inner_ref(source); is turned by the compiler into:
    let ImbrisMut {
        ptr,
        metadata,
        __marker,
    } = source;
    inner_ref(ImbrisRef {
        ptr,
        __marker,
    })
}

It is also possible for there to be multiple lifetimes in the type with some of those using exclusive reference semantics and others using shared reference semantics. The reborrowing structure can also have multiple levels of depth. An example from the Nova JavaScript engine's GcScope is given below.

struct GcMut<'a> {
    __marker: PhantomData<&'a mut GcToken>,
}


#[derive(Clone, Copy)]
struct GcRef<'a> {
    __marker: PhantomData<&'a mut GcToken>,
}

#[derive(Clone, Copy)]
struct ScopeRef<'a> {
    __marker: PhantomData<&'a ScopeToken>,
}

struct GcScope<'a, 'b> {
    gc: GcMut<'a>,
    scope: ScopeRef<'b>,
}

struct NoGcScope<'a, 'b> {
    gc: GcRef<'a>,
    scope: ScopeRef<'b>,
}

fn inner_mut(source: GcScope<'a, 'b>);

fn outer_mut(source: GcScope<'a, 'b>) {
    // inner_mut(source); is turned by the compiler into:
    let GcScope {
        gc,
        scope,
    } = source;
    let GcMut {
        __marker,
    } = gc;
    inner_mut(GcScope {
        gc: GcMut {
            __marker,
        },
        scope,
    })
}

fn inner_ref(source: NoGcScope<'a, 'b>);

fn outer_ref(source: GcScope<'a, 'b>) {
    // inner_ref(source); is turned by the compiler into:
    let GcScope {
        gc,
        scope,
    } = source;
    let GcMut {
        __marker,
    } = gc;
    inner_ref(NoGcScope {
        gc: GcRef {
            __marker,
        },
        scope,
    })
}

There is an interesting point here: the scope field can effectively be copied directly into the resulting GcScope or NoGcScope in either exclusive or shared reborrowing case, as it is Copy. Its functional record update syntax could be written out but would not make much of a point. The important bit there is that the resulting scope should not keep the original source used at all.

The gc field on the other hand needs to keep the source used. This is achieved by having the compiler mark the source.gc field as used, either as exclusive for Reborrow or as shared for CoerceShared, for the lifetime of the resulting GcMut / GcRef type.

Why recurse into non-Copy reborrowable fields?

In the above GcScope example it could be argued that GcRef should be reborrowed as exclusive by the compiler, without looking into its fields and performing the functional record update syntax on it. This would work for GcRef, and would even be more clear-cut than the destructuring since the code "turned by the compiler into", as written above, would effectively compile with minor changes (let GcMut { .. } = &mut source;) in today's Rust because all fields within GcRef are Copy but it would not mark the source as being used.

The reason for recursing becomes clearer if we take a more complicated example. Imagine a larger wrapper type with multiple reference-like fields, some of them with exclusive reference semantics, some of them with shared reference semantics, and some of the containing other wrappers that themselves contain exclusive and shared reference semantics.

struct Context<'a, 'b, 'c, 'd> {
    gc: GcScope<'a, 'b>,
    arena: &'c mut Arena,
    validity_bool: &'d AtomicBool,
}

Now if this type was made reborrowable and that reborrowing act only performed an exclusive reborrow on GcScope<'a, 'b> without looking at its fields, then the result would be a new GcScope where both of the lifetimes keep the original GcScope "used".

fn outer_mut(source: Context) {
    let Context {
        gc,
        arena,
        validity_bool,
    } = source;
    inner_mut(Context {
        gc,
        arena,
        validity_bool,
    })
}

Now if the 'b lifetime escapes in some Value<'b>, that Value would keep the gc field borrowed as exclusive. This is not what the GcScope internally defines; its internal view of the 'b lifetime is that it has shared reference semantics. This is effectively the same issue that mutating gc.gc and reading gc.scope at the same time is okay, but calling a method on gc that mutates self.scope and reading gc.scope at the same time is not okay. In the first case, only gc.gc is used exclusively and gc.scope is used as shared, whereas in the latter case the whole gc is used exclusively and gc.scope is used as shared, which is an aliasing violation.

The correct formulation for transitive reborrowing is then this: for each field in the reborrowed type that is !Copy, recurse into the type and check if it has any !Copy fields. If it does, recurse into it. If not, then mark that field used. An example using Context is given below.

fn outer_mut(a: Context) {
    // `Context: !Copy` so recurse.
    let Context {
        // `GcScope: !Copy` so recurse.
        gc,
        // `Arena: !Copy` but has no fields: mark as used.
        arena,
        // `&T: Copy` so skip.
        validity_bool,
    } = source;
    let GcScope {
        // `GcMut: !Copy` but has no `!Copy` fields: mark as used.
        gc,
        // `ScopeRef: Copy` so skip.
        scope
    } = gc;
    let GcRef {
        __marker,
    } = gc;
    inner_mut(Context {
        gc: GcScope {
            gc: GcMut {
                __marker,
            },
            scope,
        },
        arena,
        validity_bool,
    })
}

Reference-level explanation

core libs

Two new traits, Reborrow and CoerceShared, are added to and exposed from core::ops.

pub trait Reborrow {}

pub trait CoerceShared: Reborrow {
    type Target: Copy;
}

A blanket implementation is provided for any type that implements Copy:

impl<P: Copy> Reborrow for P {}

impl<P: Copy> CoerceShared for P {
    type Target = Self;
}

The traits are also implemented for &mut T and &T:

impl<T: ?Sized> Reborrow for &mut T {}

impl<T: ?Sized> CoerceShared for &mut T {
    type Target = &T;
}

Blanket implementations for Pin<&mut T> and Option<&mut T> are also added:

impl<T: ?Sized> Reborrow for Pin<&mut T> {}

impl<T: ?Sized> CoerceShared for Pin<&mut T> {
    type Target = Pin<&T>;
}

impl<T: ?Sized> Reborrow for Option<&mut T> {}

impl<T: ?Sized> CoerceShared for Option<&mut T> {
    type Target = Option<&T>;
}

Compiler changes: method probing

The existing Rust reference section for method calls describes the algorithm for assembling method call candidates, and there’s more detail in the rustc dev guide.

The key part of the first page is this:

The first step is to build a list of candidate receiver types. Obtain these by repeatedly dereferencing the receiver expression’s type, adding each type encountered to the list, then finally attempting an unsized coercion at the end, and adding the result type if that is successful. Then, for each candidate T, add &T and &mut T to the list immediately after T.

We add to the "for each candidate T" list expansion the step that if T: CoerceShared where <T as CoerceShared>::Target is not equal to T, and T is not equal to some &mut U then <T as CoerceShared>::Target is added to the list immediately after &mut T. Note: the condition on T not being some &mut U is because of &mut T: CoerceShared<Target = &T>, which would otherwise produce unnecessary duplicates.

This does change the list of candidate receiver types, adding shared coercion targets to it when applicable. These should be rare, so the effective change is minimal.

Compiler changes: reborrow adjustments

In the compiler, when resolving coercion a new possible reborrow adjustment is added. The condition for performing this coercion is that the source type is Reborrow: !Copy and the target type is equal to the source type, or that the source type is CoerceShared: !Copy and the target type is equal to the source type's CoerceShared::Target type. In the first case an exclusive reborrow adjustment is inserted, and in the second case a shared reborrow adjustment is inserted. This replaces the PinReborrow adjustment currently in place for Pin<&mut T> reborrowing specifically.

This adjustment creates a new target type struct by performing the following steps with the source type and the target type as its parameters:

1. Let should_mark be true.
2. For each field in the target type, do:
   1. If the target field type is Copy and the corresponding source field type is Copy, then
      1. Assert that the two types are the same.
      2. Perform a copy from source to target.
   2. If the target field type is Copy and the corresponding source field type is !Copy, then
      1. Assert that the reborrow adjustment kind is "shared".
      2. Assert that the source field type implements CoerceShared and its CoerceShared::Target is equal to the target field type.
      3. Set should_mark false.
      4. Note: marking should_mark false is superfluous, we could mark any number of reborrowable source fields used as shared.
      5. Recursion: perform these steps (starting at the top) with the source field type and the target field type as parameters.
      6. Note: the recursion is needed for handling changes in memory layout between source and target field types.
   3. If the target field type is !Copy and the corresponding source field type is !Copy, then
      1. Assert that the reborrow adjustment kind is "exclusive".
      2. Assert that the two types are the same.
      3. Set should_mark false.
      4. Recursion: perform these steps (starting at the top) with the source field type and the target field type as parameters.
      5. Note: the recursion is needed for ensuring that only necessary parts of the source type are reborrowed as exclusive.
3. If should_mark is true, then
   1. If the reborrow adjustment kind is "shared", then
      1. Mark the the source type used as shared.
   2. If the reborrow adjustment kind is "exclusive", then
      1. Mark the source type used as exclusive.

Drawbacks

The main drawback of autoreborrow traits is the conversion from the T to <T as CoerceShared>::Target and the related addition to method probing logic. This adds a new piece to an already complicated Deref coercions set. Reborrowable custom reference types are also relatively niche, though very useful.

The algorithm for the compiler to crunch through reborrowed vs copied fields is not too bad, but it's still a bit involved and if it were to be reproduced at each reborrowing coercion site then the performance penalty would probably be significant. Luckily, the results of the algorithm never change so it should be fairly easy to effectively cache the results.

Aside from that, reborrowing is not too complex and rustc already has existing code for autoreborrowing Pin<&mut T>, which this would supersede.

Rationale and alternatives

Rationale

The rationale for choosing a single trait with a single target GAT and no methods is as follows:

• Reborrowing only makes sense for exclusive reference types: reborrowing of shared reference types as shared is exactly equal to copying the shared reference type.
• An exclusive reference should only be exclusively reborrowable into itself, ie. &mut T -> &mut T or MatMut<'_> -> MatMut<'_>. A &mut T -> &mut U should be thought of as a reborrow followed by a dereference. Hence, a separate Reborrow::Mut GAT is not needed.
• An exclusive reference type that doesn't need to be reborrowable as shared may exist: the proposed Reborrow trait does not directly support this, but it can implemented by creating a dummy type (with same layout as the exclusive reference) which is never used and has no methods on it. The exclusive reference type will thus be reborrowable as shared, but into a type that cannot be used for anything.
• Adding a method to reborrowing would make all coercion sites into implicit injection points into user-code. This seems both too powerful, too magical, and too subtle to allow, hence no method is proposed.

Compiler-level reasoning around custom type autoreborrowing

Exclusive reborrowing of custom types is fairly straightforward as the only complexity lies in implementing the lifetime replacement and extension, and the disabling of the source T. This is already implemented in the compiler for Pin<&mut T> specifically, and extending this for all types implementing the Reborrow trait ought not be too complicated.

Shared reborrowing is much less so, as it requires the compiler to copy the source T into <T as Reborrow>::Ref. The usual case for reborrowable types is that they have only one (or no) data field of the same kind for both the exclusive and shared reference types, meaning that a transmute would be trivially sound (at least for #[repr(transparent)] types), but for multi-field types the soundness of the transmute cannot be guaranteed. A basic reasoning mechanism safe transmutes is provided by the safe transmute project, which can detect grievous mistakes like transmuting padding bytes into data. More complex reasoning, like making sure that two usize fields do not logically flip in the transmute, cannot be performed by the compiler and would require #[repr(C)] usage on both sides of the transmute. As this requires manual checking by the user of the Reborrow trait, the trait is proposed as unsafe with the requirement being that T is transmutable into T::Ref. Hence, the trait's safety requirements are inherited from std::mem::transmute.

Mitigating the unsafety of an injected transmute

As the transmuting of T to T::Ref is unsafe and requires manual checking, we'd like to avoid the injected transmute. The obvious thing to do would be to add a method fn reborrow(self) -> T::Ref (note: the method must take self by value, not by reference) to the Reborrow method but as discussed above, this is not proposed because it would mean injecting implicit user-controlled code with arbitrary side-effects to every coercion site. This runs counter to what reborrowing is generally though of being, "copy of a pointer with extra lifetime analysis", and we'd thus want to avoid it. (Contrast this with adding an fn copy() method to the Copy trait.)

With the safe transmute project, the implicit transmute could be improved by requiring a type Ref: TransmuteFrom<Self, _> trait bound on the GAT: with this the source and target would at least be confirmed to be safely transmutable. As mentioned earlier, this would still not guarantee that the transmute is sound but would at least go a long way in mitigating the issue.

An alternative to an explicit reborrow method would be to add a type Ref: From<T> bound to the GAT and have the compiler inject a <T as Reborrow>::Ref::from(T) call at the reborrow site. The downside here is that the From impl is still user-controlled code and could likewise have arbitrary side-effects. That being said, From trait implementations are well understood (as compared to a custom reborrow method, or even eg. Deref vs AsRef vs Borrow), are held to a high standard, and misusing said traits is likely to cause various other issues; they are thus unlikely to contain side-effectful code.

Finally, in the future if/when const fns in traits become available, improvements on the above method-based approaches appear: the custom reborrow method could be made into a const fn reborrow() or, if trait bounds requiring const implementations of traits or trait methods become possible, the From bound could be turned into Target: const From<Self>::from. With this the fear of entirely arbitrary side-effects is further mitigated, though not entirely removed.

The safe transmute trait bound would not be enough for the Reborrow trait to be marked as safe, due to the soundness issue. Hence, it is not a complete mitigation.

Of the possible complete mitigations, this RFC considers only the type Ref: From<T> bound worthy of further study. An explicit reborrow method would have high chance of misuse, but a From trait conversion can be fairly well trusted to be valid and safe to call implicitly. Adding const bounds would restrict the possible misuse of a reborrow or from method, but as the capabilities of const expand the restrictions placed by this bound would slowly erode. Hence, those bounds would not really be reliable soundness guarantees.

Detecting the reborrowed lifetime

In the above reborrow formulation, the Reborrow trait itself has a lifetime parameter that the compiler uses to detect which lifetime is being reborrowed, in case there are multiple (or none). This seems like a neat solution, but if it is deemed problematic then an alternative would be to require that reborrowable types and their Ref targets have exactly one lifetime.

This would cause some trouble for some users (including the author's GcScope), but if it is the price that has to be paid for true reborrowing of custom types, then so be it.

Alternatives

Alternative 1: Reference-based method(s)

The most obvious alternative to the proposed Reborrow trait would be to take what we already have in userland and make it part of the core library and rustc. This would mean implementing a trait or traits in the vein of the reborrow crate:

/// Shared reborrowing.
trait Reborrow<'short, _Outlives = &'short Self> {
    type Target;

    #[must_use]
    fn rb(&'short self) -> Self::Target;
}

/// Exclusive reborrowing.
trait ReborrowMut<'short, _Outlives = &'short Self> {
    type Target;

    #[must_use]
    fn rb_mut(&'short mut self) -> Self::Target;
}

Note that two traits are given here to match what the reborrow crates does, not as an explicit alternative proposal.

The main benefit with this approach is that this follows the current userland state-of-the-art way of creating "reborrow-like" functions and would effectively bless a particular, well-liked userland implementation into the language together with compiler support for automatically injecting the trait method calls, relieving the user of having to call the methods manually.

The main issue with this approach is that this does not implement true reborrowing: the 'short lifetime gets bound to the temporary borrow created when calling these methods, meaning that lifetime extension does not work out of the box. The 'short lifetime is a strict subset of the source lifetime (call it 'long), instead of being able to extend to equal it. To overcome this issue, rustc would need to special-case this trait implementation to allow lifetime extension of 'short into 'long. But doing so is unsound!

As the implementation of these methods is user-controlled, users often find neat ways to stretch implementations beyond the limits of what was intended but not explicitly made impossible. In a pure reborrow frame of mind, turning &'short Self<'long> into Self<'short> with 'short being allowed to lifetime extend to equal 'long seems sound, but it is not. The &Self reference is a pointer onto the stack (most likely) and allowing lifetime extension from Self<'short> to Self<'long> would mean that rustc would allow the &Self reference to escape its call stack. This is equal to returning a pointer pointing to released stack memory, ie. undefined behaviour.

Thus, in current Rust syntax a Reborrow trait taking self by reference while allowing lifetime extension would be unsound. To solve this issue, a new syntax for taking a value by reference that cannot be used as a place or turned into an address would be needed, perhaps something like:

trait Reborrow<'short, _Outlives = &'short Self> {
    type Target;
    fn reborrow(^'short self) -> Self::Target;
}

trait ReborrowMut<'short, _Outlives = &'short Self> {
    type Target;
    fn reborrow_mut(^'short mut self) -> Self;
}

As this sort of a reference would be something entirely new (though it probably relates to Pin<& (mut) T> somehow), it is deemed too complicated a change for this RFC. Because true reborrowing is one of the focal aims of this RFC, any reference-based trait method is thus considered a non-starter due to the soundness issue.

Alternative: Generic type parameter

The Reborrow trait (any version of it) could be defined with a generic type parameter. This would be more generic than having a generic associated type, though it would likely make the implementation correspondingly more complex as well. This doesn't help with the soundness issues of a reference-based method or anything else related to the trait. It also has a new issue lurking within it: a Reborrow trait that can be implemented multiple times over gives users the power needed to implement automatic type coercion. Eg. for integer coercion, you need only to implement Reborrow<uN> for MyInt and Reborrow<iN> for MyInt for all N of your preference and now your MyInt type will be reborrow-coerced into whatever integer type is needed by a given call.

For this reason, this RFC does not propose a Reborrow trait with a generic type parameter.

Alternative: Special reborrow syntax

If userland reborrowing was made explicit in the source code using special syntax, then the compiler would not need to implicitly inject user-controlled code at coercion sites. This would make user-controlled reborrow methods perfectly acceptable, as they would be visible in user code.

The ergonomic clones feature has added a foo.use suffix operator for (effectively) calling clone() in an ergonomic way. A similar thing could be done for reborrowing using eg. the ref and mut keywords:

fn mixed_reborrowing(source: MyMut) {
    let result = source.mut.mutate();
    result.ref.observe();
    let result1 = source.ref.observe();
    let result2 = source.ref.observe();
    result1.ref.observe();
    result2.ref.observe();
    source.mut.mutate();
}

This does not solve the problem of lifetime extension in any way, ie. this does enable implementing true reborrowing. It also takes away the "auto" from "autoreborrowing". (The ref and mut suffix syntax might also be better used for other purposes, such as generalised Deref / DerefMut or AsRef / AsMut calls.) For these reasons, this RFC does not propose adding a custom syntax for reborrowing.

Alternative: "Event handler" methods

The soundness issue related with returning a new value from a reference-based reborrow method makes reference-based reborrow methods a non-starter. If user-controlled code during reborrowing is still wanted, an alternative to the conversion methods is to have "event handler" methods like on_reborrow and on_reborrow_mut. These methods would take the reborrow result by reference but would have no return value, thus avoiding the soundness issue at the cost of still requiring the compiler to inject bitwise copies and transmutes.

These would allow compiler-injected reborrowing to trigger user-controlled code, which would be beneficial for eg. reference counted types. This would be useful for eg. RefCell<T>'s Ref<'a, T> and RefMut<'a, T> types, which perform (cheap) reference incrementing/decrementing on Clone and Drop. If the reference counting of these types could be performed as part of reborrowing, then methods like RefMut::map_split could be generalised over both reference counting and reborrowing types.

The downside here is that cloning is explicit, and this explicitness and clippy linting makes it easy for users to opt out of cloning when it is not needed. With compiler-injected reborrowing, this becomes much more complicated. Consider the following code:

fn method(r: RefMut) {
    method_a(r);
    method_b(r);
}

In current Rust, this will not compile as r is moved into the method_a call. To fix this, an explicit .clone() must be inserted:

fn method(r: RefMut) {
    method_a(r.clone());
    method_b(r);
}

Now imagine the original code being compiled with RefMut implementing Reborrow (and still being Drop like previously): both method calls will implicitly reborrow r, effectively performing two clone calls and one drop:

fn method(r: RefMut) {
    method_a(r.reborrow()); // calls on_reborrow() on the result
    method_b(r.reborrow()); // cals on_reborrow() on the result
    // implicit `drop(r);` happens here
}

A sufficiently smart compiler could theoretically optimise the extra clone out, but because of the on_reborrow method it would be very hard or impossible to prove that the combination of r.reborrow() and drop(r) is equal to moving r into the second call. The "ergonomic clones" RFC does include a provision for allowing the compiler to optimise out extra clones; the same kind of provision could be asserted for Reborrow to allow the compiler to remove the extra calls, but it is an added complexity.

Therefore, this RFC does not propose any on_reborrow like trait methods and instead suggests keeping reference counting and reborrowing separate from one another.

Related efforts

Generalised Deref and DerefMut traits

The Deref and DerefMut traits are quite magical and powerful, but they are not quite powerful enough to implement the userland reborrow-like functions because they must always return references. A generalised Deref(Mut)Generalised trait would be one where the return value is the Target GAT directly and not a reference of it. These traits would be strong enough to replace the reborrow crate and would have the benefit that a DerefGeneralised trait enables passing reborrowable parameters with a simple &my_ref or &mut my_mut to explicitly trigger dereferencing. This would be visible in the source code (usually; method calls wouldn't show them) and thus the trait methods would not be as scary, compared to implicitly inserted user-controlled code.

Again, these traits on their own do not and cannot solve the lifetime extension issue. In this sense, the current userland's "reborrow-like" functions could actually be more accurately called "generalised deref functions" as the two are exactly equivalent.

Reborrowing of reference counted references

Briefly discussed above in the "event handler" methods alternative, this section expands on how and why reference counted references might want to tap into method-based reborrowing, and why this is probably not the best of ideas.

The standard library's RefCell<T> gives out Ref<'a, T> and RefMut<'a, T> custom reference types; each of these carries with it a pointer to the reference counted data. Refs are Clone and increment the RefCell's ref counter, while RefMut have an internal clone method that gets used in eg. the RefMut::map_split method and increments the RefCell's mut counter. Both types then have Drop impls that decrement their respective counters again at the end of their lifetimes. These types of references are not reborrowable in the traditional sense as they need to run custom code during reborrowing, but they are reborrowable in the sense that a Ref or RefMut created based on an earlier RefMut should be allowed to outlive its source.

eg. Given the following kind of code:

fn example() {
    let rc = RefCell::new(0u32);
    let result = {
        let ref_mut = rc.borrow_mut(); // mut counter +1
        {
            let (ref_mut_1, ref_mut_2) = ref_mut.map_split(...); // mut counter +1
            drop(ref_mut_1); // mut counter -1
            ref_mut_2
        }
    };
    println!("ref_mut_2: {:?}", result);
    // mut counter -1
}

On the type level, the ref_mut is moved out of in map_split, meaning that ref_mut_1 and ref_mut_2 are both dependent on a borrow that, in a sense, dropped already. Despite this, they can not only be used but also returned from their current scope and moved to parent scopes up until the scope where the originating RefCell is.

This is very similar to how reborrowed references can use lifetime extension to return to their "true source of origin" despite being borrowed deep inside a call graph. If RefMut's clone method were to be rewritten in terms of a Reborrow trait with a method incrementing the mut counter, and if the Reborrow trait could be extended to allow types containing data with Drop impls, then the RefMut::map_split could be generalised further.

There is a complication to this: since the borrow: BorrowRefMut field of RefMut implements Drop, the above code could see three drop calls instead of the current two. Right now ref_mut is never dropped as it is moved out of and into map_split but if that move was made into a reborrow, then an extra reborrow-injected clone would happen at the map_split call site and an extra drop would happen when the scope in which ref_mut was create is left. As an optimisation, the compiler could choose to only perform a reborrow when a move isn't possible, but that might be very complicated to implement.

Prior art

• Formalise reborrows RFC: rust-lang#2364
• Reborrow trait pre-RFC: Rust Internals Forum #20774
• Feature request to simulate exclusive reborrows in user code: rust-lang#1403
• Experimental support for Pin reborrowing: rustc#130526
• Obscure Rust: reborrowing is a half-baked feature haibane_tenshi's blog

Unresolved questions

Matching the custom type's lifetimes to the reborrowed lifetime can be done via a lifetime bound on the trait, but it is a magical feature that doesn't really otherwise have much of a use. Whether this matching could be done in a better way is an unresolved question, possibly in a manner allowing multiple reborrowed lifetimes if that happens to be reasonable as that would make passing of "collection of exclusive references" kind of context structs easier. This is not considered a high priority for this RFC, however.

Future possibilities

It seems probable that autoreborrowing will open wide the doors for AsRef, Deref, Borrow, and similar traits to be re-examined and extended in a world where custom reference types are commonplace. As an example, currently a &mut T can be passed to a method that takes &mut U where T dereferences into a U. With the proposed Reborrow trait, this would not be possible. A Reborrow-aware generalised Deref trait that enables this would need to take the custom reference type T by value and produce a U as a result, ie. an Into trait. Allowing Into::into calls to be injected at reborrow sites seems like a possibly bad idea, and is thus not proposed in this RFC. Revisiting this is, however, possibly worthwhile at a later time if eg. ^(mut)-like references that cannot be used as places make it into the language.

Additionally, some custom reference types require a context parameter for performing actual dereferencing. Far into the future, a true generalised Deref trait might thus look something like this:

impl<'a, T> DerefGeneralised for ArenaHandle<'a, T> {
    type Context = &'a Arena;
    type Target = &'a T;

    fn deref_generalised(^self, ctx: Self::Arena) -> Self::Target {
        // ...
    }
}

But, as the traits become more generic they become more and more complex and error-prone, and thus it is unclear if these sorts of generalisations truly deserve to make it into the language or if they should forever stay in userland only.

[kpreid]

@aapoalas

unsafe trait Reborrow<'a>: !Clone + !Drop {
    type Ref: Copy;
}

Comment #1: On first read, I had trouble noticing that the idea here is that exclusive-to-exclusive reborrows use Self-with-lifetimes-changed, as opposed to “okay, this is the shared reference version; where is the exclusive reference one?” I would suggest adding a comment to help people at least notice the confusion; something like:

unsafe trait Reborrow<'a>: !Clone + !Drop {
    type Ref: Copy;  // but no type Mut
}

---

Comment #2: The trait being unsafe will make it somewhat undesirable to implement even when it would be correct. One thing that comes to mind is: if the type in question has no uses via shared reference (I have a case like this; the type is essentially pointers and context for writing to a complex buffer), then it doesn't need a Ref type for shared reborrowing. And the unsafety is because type Ref can't be proven correct; therefore, it would be useful if there was a way to safely implement exclusive-only reborrowing.

This could be done using a derive macro to generate the unsafe impl together with the “dummy type (with same layout as the exclusive reference) which is never used and has no methods on it” you mention, but it's inelegant to require the dummy type to exist.

I think this is a sufficient reason to consider splitting it into two traits, so exclusive-only reborrows can be safe to implement. (Reborrow and ReborrowToShared? ReborrowRef?)

[CAD97]

I concur that the exclusive reborrow should be a safe trait. The prior art here is Copy, actually — there are unsafe requirements for impl Copy to be valid, but the compiler checks those for you. For Copy, it's for all fields to be Copy; for Reborrow, it's for all fields to be Copy or Reborrow.

The Reborrow trait specifying which lifetime will be covariantly reborrowed is convenient for definition, but seems unnecessary. A reborrow really is halfway between a move and a copy — the conversion from place to value shortens any covariant lifetimes as is necessary, and for impl Reborrow types, the place is made accessible again after the produced value is expired. (impl Copy makes it immediately usable.)

What this limits is types which are logically covariant but the implementation is invariant, such as current io::BorrowedCursor. Reborrow could be the way to make such types functionally covariant, but due to the limitation to a single lifetime and ability to have the compiler check immediate safety of reborrowing covariant lifetimes, I think that unsafely forcing covariance should be orthogonal to Reborrow.

(That said, if a type has multiple covariant lifetimes, there's an argument to be made that said lifetimes can and should be unified immediately into a single lifetime. I'm not fully confident that this is always the case, though. But safe Reborrow and the real benefit of orthogonality are sufficient reason on their own, IMHO.)

Contrary to the requirement in the current RFC draft, I don't think Reborrow should require the absence of Copy; rather, I think Copy should actually imply Reborrow due to this way of viewing the operation. When you think about it this way, it's perfectly valid to use any Copy value as if it were just Reborrow, the same way that it's valid to use it as just Clone.

However! This is if the Reborrow trait makes no mention of Deref in its definition. It might need to, if we want my_mut.f() to find fn f(self: MyRef<..>) during lookup and be able to call it. Method lookup with custom receivers is complicated and I don't really fully understand it, but it's clear that a type-adjusting reborrow splits the lookup space if it's allowed to nonterminally adjust the deref output type. I don't think it's needed^[1], but I'm not fully confident in asserting such.

Note also that, while also often called reborrowing, &'a mut T to &'b T is meaningfully different from &'a mut T to &'b mut T. Notably, while there is an interest in potentially relaxing this restriction, mut reborrowing can effectively happen in any covariant position (e.g. &&mut), whereas ref reborrowing can only happen by value.

That type-adjusting reborrowing can only happen by value I think is a very helpful restriction here; due to this restriction, the safety of the reborrow can be checked by the compiler^[2] and not require shared layout^[3]. Specifically, the compiler can check it can reborrow fields name-wise (including privacy access checks at the scope of the impl) like FRU does, but ignoring any PhantomData-typed fields^[4], and ofc also allowing the type to be adjusted, unlike FRU.

That's my idea of semantics, but what about how to express the difference between reborrow as Self and type adjusting reborrowing? I concur that a split between the two makes sense. The current coercion traits are Deref[Mut] (autoderef), Unsize (unsizing the type tail), and CoercePointee (types that can unsize a generic, in self rather than be unsized themselves, which is an automatic Unsize impl).

So I propose Reborrow for the as-Self and CoerceShared for type-converting operation, at least for this stage in the bikeshedding. Thus my concept of Reborrow would look roughly like:

// in std::ops

#[lang = "reborrow"]
pub trait Reborrow {}

#[lang = "coerce_shared"]
pub trait CoerceShared: Reborrow {
    type Output: Copy;
}

// blanket impls

impl<T: Copy> Reborrow for T {
    // SAFETY: where T: Copy, reborrow and copy are identical
}

impl<T: Copy> CoerceShared for T {
    // SAFETY: reborrowing as Self is definitionally possible
    type Output = Self;
}

// helper pseudo macro for exposition purposes

macro reborrow_as($T:ty, $expr:expr $(,)?) {
    macro where $T = PhantomData<_> {
        PhantomData
    } else {
        $expr
    }
}

// user code

impl<T: ?Sized> Reborrow for MyMut<'_, T> {
    // roughly, automatic obligation:
    const _: () = assert!(!needs_drop::<Self>())
    where $( ${Self.field:ty}: Reborrow, )*;
}

impl<'a, T: ?Sized> CoerceShared for MyMut<'a, T> {
    type Output = MyRef<'a, T>;
    // roughly, automatic obligation:
    const _: fn(Self) -> Self::Output = |_self| MyRef { $(
        ${MyRef.field:ident}: reborrow_as!(
            ${MyRef.field:ty},
            _self.${MyRef.field:ident},
        ),
    )* }
}

---

1. Calling &self methods through &mut impl Deref doesn't reborrow &mut as &, it just uses &mut: Deref. So I think it's sufficient to get intuitive behavior to check type-converting-reborrowed self for being a custom receiver after checking self and before checking <Self as Deref>::Output and never check the type-converting-reborrowed type's Deref chain, even if it's different. But it's almost certainly in error to provide type-converting-reborrows that have differing Deref::Output, so that's worth a warning. ↩︎

2. Or, well, at least be equivalent to entirely safe code, justifying being "inside the safety boundary" and thus not requiring unsafe despite potentially being unsound in combination with other likely to exist unsafe code around custom ref-like types. ↩︎

3. Although, the shared representation restriction isn't that big a deal in practice. It doesn't require that you sacrifice any repr(Rust) benefits; an idiomatic way to set up ref-like types is to have a raw Ptr type and #[repr(transparent)] struct Ref(Ptr) wrappers of the raw ptr-like type for your ref-like types. ↩︎

4. The nightly derive(CoercePointee) privileges PhantomData like this already, so this isn't a new concept just for Reborrow. ↩︎

[kpreid]

@CAD97

(That said, if a type has multiple covariant lifetimes, there's an argument to be made that said lifetimes can and should be unified immediately into a single lifetime. I'm not fully confident that this is always the case, though. …

Multiple covariant lifetime parameters can be merged into one only when no references are taken out using the longer lifetime, which is a common case, but not universal. Here is a counterexample:

#[derive(Clone, Copy)]
struct Lookup<'o, 'i> {
    outer: &'o [String],
    inner: &'i [usize],
}

impl<'o> Lookup<'o, '_> {
    fn lookup(&self, ii: usize) -> &'o str {
        &self.outer[self.inner[ii]]
    }
}

fn use_lookup_with_temp_inner<'o>(outer: &'o [String]) -> &'o str {
    let inner = &vec![0, 1, 2, 3];
    let lookup = Lookup { outer, inner };
    lookup.lookup(0)
}

If Lookup is changed to use a single lifetime parameter, then writing use_lookup_with_temp_inner is impossible — it needs &'o str but can't get it.

[beepster]

The guide-level explanation section feels like it's actually the motivation section. I think it would be a good idea to move its contents there.

[Imbris]

I do have a case where the exclusive reference has an extra field that isn't in the shared reference which is probably relevant wrt whether transmutation would work there. I mention it a bit here #t-lang > autoreborrow traits @ . It's not open source so I can't link the actual usage but the extra field is essentially a pointer to some state used for mutation tracking and allocation in the associated arena. It's mainly an optimization so I'm sure I could find workarounds and also just exclusive autoreborrowing would already be very useful in this case.

[aapoalas]

Thank you for the comment! Very good point about me not being explicit about the trait implicitly defining the exclusive reborrowing. The point about exclusive-to-exclusive being safe and only exclusive-to-shared requiring unsafe makes me think that despite it being a bit more complex, two traits does sound like the right choice.

I'll change the draft to propose two traits, one safe and entirely empty, and an unsafe one with a GAT for reborrow-as-shared.

[aapoalas]

Hello, and thank you for the great comment and deep insight! This is much appreciated! <3

@CAD97

The Reborrow trait specifying which lifetime will be covariantly reborrowed is convenient for definition, but seems unnecessary.

I wonder if this is truly the case. My understanding around lifetime variance is not quite enough to fully understand the entirety of this on an intuitive level. In my use case, I have a GcScope<'a, 'b> ZST marker type where the first lifetime derives from an exclusive source and the second derives from a shared source. When this struct is reborrowed as exclusive, anything observing the 'a lifetime should invalidate but anything depending on the 'b must not.

On a theoretical level, this should not be an issue but I do expect that it would be: imagine a method that takes GcScope, and calls another method that also takes a GcScope that then returns a value bound to the 'b lifetime (I have a fair bit of such methods.). Now what should happen is that the original GcScope becomes reusable when the method returns, since the 'b lifetime does not participate in the reborrowing but is purely passed through. But if Rust was injecting reborrowing for each lifetime, then it would have to assume that the 'b lifetime has been reborrowed as exclusive and thus the original GcScope cannot be reused until the return value is dropped.

I do mention in the RFC that this sort of mixed use could be first on the chopping block, though; I'll then just have to live with two separate ZST marker types being passed through my whole call graph

@CAD97

Contrary to the requirement in the current RFC draft, I don't think Reborrow should require the absence of Copy; rather, I think Copy should actually imply Reborrow due to this way of viewing the operation. When you think about it this way, it's perfectly valid to use any Copy value as if it were just Reborrow, the same way that it's valid to use it as just Clone.

This sounds reasonable, and would be a nice path to a derivable Reborrow trait. I'm only a slight bit worried about explosions in the compiler's search space... but since I don't exactly know how the compiler works hereabouts (I've only really been looking at the Pin<&mut T> reborrow implementation for reference.), that is basically just me being afraid of my own shadow

@CAD97

However! This is if the Reborrow trait makes no mention of Deref in its definition. It might need to, if we want my_mut.f() to find fn f(self: MyRef<..>) during lookup and be able to call it. Method lookup with custom receivers is complicated and I don't really fully understand it, but it's clear that a type-adjusting reborrow splits the lookup space if it's allowed to nonterminally adjust the deref output type. I don't think it's needed^[1], but I'm not fully confident in asserting such.

This ... is actually an interesting consideration. I do absolutely think that reborrowing should participate in self method lookup, and indeed Deref::Target and CoerceShared::Output differing from one another should definitely be an error. And it should preferably be functionally equivalent to have &MyMut or MyRef, the difference being at most lifetime narrowing. (See: "true reborrowing" vs current fascimile approaches in userland.)

So what if CoerceShared (I liked the name, I'm yoinking it, thank you very much!) did actually require Deref and borrowed its Target type for its output...?

unsafe trait CoerceShared: Reborrow + Deref {
  // implied output type is Target where Deref::Target: Copy;
}

This both explains reborrowing to the Deref chain resolution logic (or close enough to be interesting at least), and almost-but-not-quite explains the memory layout requirements of a safe CoerceShared: it must be possible to go from a reference to the exclusive reference type, and find a reference to the shared reference type. This probably has some holes, since eg. Box<u32> could Deref to u32 and thus CoerceShared from Box<u32> to u32 would be a possible impl (ignoring all other requirements, like !Clone). That's of course obviously not going to work, not without an explicit method, so this is not enough to say CoerceShared is a safe trait. But it's pretty close: if eg. the compiler could check that the Deref::deref method impl is equal to a copy of the shared reference, going from &T to &T::Target, then that'd actually be sufficient to remove the unsafe.

This would also unify the "generalised Deref" and "reborrowing" notions that have been a bit of a thorn in my side with preparing the draft. (Or at least it would bring them closer together.) I'm beginning to think that this is a direction I should strongly consider, possibly even promote as the foremost proposal. Thank you, this was a great thing to bring up!

@CAD97

That type-adjusting reborrowing can only happen by value I think is a very helpful restriction here; due to this restriction, the safety of the reborrow can be checked by the compiler^[2] and not require shared layout^[3]. Specifically, the compiler can check it can reborrow fields name-wise (including privacy access checks at the scope of the impl) like FRU does, but ignoring any PhantomData-typed fields^[4], and ofc also allowing the type to be adjusted, unlike FRU.

Sorry, I didn't understand what "FRU" is, could you open up the abbreviation?

Can the compiler actually reason about Imbris' case of differing fields between exclusive and shared reference types? Of course, going from a larger struct to a smaller struct is theoretically pretty easy; even if the fields don't happen to align so that the result is a single memcpy, it's still a relatively easy "copy only some fields over, possible reorder" operation. The only thing that really sticks out to me is the requirement of borrowing fields name-wise: is that really something the compiler would accept? Of course, if it can be checked at compile time that all field names in the shared reference struct match a field in the exclusive reference struct, and that their types match or can be reborrowed from one to the other, then that's not a hard or unreasonable requirement for users to fulfill. It just feels a little "unrefined", somehow

---

1. Calling &self methods through &mut impl Deref doesn't reborrow &mut as &, it just uses &mut: Deref. So I think it's sufficient to get intuitive behavior to check type-converting-reborrowed self for being a custom receiver after checking self and before checking <Self as Deref>::Output and never check the type-converting-reborrowed type's Deref chain, even if it's different. But it's almost certainly in error to provide type-converting-reborrows that have differing Deref::Output, so that's worth a warning. ↩︎

2. Or, well, at least be equivalent to entirely safe code, justifying being "inside the safety boundary" and thus not requiring unsafe despite potentially being unsound in combination with other likely to exist unsafe code around custom ref-like types. ↩︎

3. Although, the shared representation restriction isn't that big a deal in practice. It doesn't require that you sacrifice any repr(Rust) benefits; an idiomatic way to set up ref-like types is to have a raw Ptr type and #[repr(transparent)] struct Ref(Ptr) wrappers of the raw ptr-like type for your ref-like types. ↩︎

4. The nightly derive(CoercePointee) privileges PhantomData like this already, so this isn't a new concept just for Reborrow. ↩︎

[aapoalas]

Thank you, I'll take a look at moving it.

[aapoalas]

@CAD97 Assuming that "symbolic execution" of the Deref::deref method is at least theoretically possible, I think you have the Right And True, Safe Reborrowing of our dreams described in your code. The only addition I think is a blanket impl on Deref!

impl<T: Reborrow + Deref> CoerceShared for T where T::Target: Copy {
    type Output = T::Target;
}

With this we first have Copy traits implementing Reborrow as fait accompli, as you saw. Then, for (simple) custom exclusive reference types that are currently move-only, we have them define Deref to show the compiler how to find the offset in the type to copy the corresponding custom shared reference type's data from. Finally, for reborrowable structs we define their Reborrow and CoerceShared as a field-wise operation: here it is worth it to note that any Copy fields are always simply copied over without reborrowing. This gives us automatic handling of structs with some fields being exclusive and others being shared reborrows, like my GcScope.

EDIT: There's one issue playing with this on the playground: the Deref and Copy blanket impls conflict.

[CAD97]

@aapoalas

But if Rust was injecting reborrowing for each lifetime, then it would have to assume that the 'b lifetime has been reborrowed as exclusive

You can't borrow a shared lifetime exclusively. That's fundamentally not a thing.

When you reborrow Scope<'a, 'b> where both lifetimes are covariant, you get Scope<'m, 'n> where 'a: 'm, 'b: 'n. What exactly that means depends on what set of loans those lifetimes represent. If 'a captures any exclusive loans, then using any type mentioning 'a (or otherwise invalidating 'a) must come after invalidating 'm. The same goes for 'b and 'n. But if the lifetime only captures shared loans, then the parent lifetime can be used without invalidation of the child lifetime.

This, exactly, already happens whenever you turn a place with type Scope<'a, 'b> into a value (rather than borrow it). What's missing is the ability for the compiler to return access to the source place after the child lifetimes expire, as the only option for user defined types is move or independent copy.

Consider also that fn(&mut T) -> Type<'_> and fn(&T) -> Type<'_> produce the "same" type with the "same" lifetime position, but each captures a distinct loan set, so the former prevents access to the input reference, but the latter doesn't.

@aapoalas

Sorry, I didn't understand what "FRU" is, could you open up the abbreviation?

Functional Record Update — it's the Type { ..val } syntax.

@aapoalas

The only thing that really sticks out to me is the requirement of borrowing fields name-wise: is that really something the compiler would accept?

The compiler can do anything. FRU does a name-wise pairing already… or well, it pairs based on identity, since it can't change the type. Or actually, type adjusting FRU has been RFC-accepted; it's acceptable to use FRU to fill out Type<U> from Type<T>. CoerceUnsize does the same thing, changing the one pointer field from thin to wide.

Extending this for CoerceShared to do a kind of FRU between different type stems doesn't seem too out there to me, but I'm not on T-lang, so I don't have any actual say.

@aapoalas

we have them define Deref to show the compiler how to find the offset in the type to copy the corresponding custom shared reference type's data from

See how derive(CoercePointee) solves that exact thing. You mark the #[pointee] type parameter and are required to only have a single field mention that parameter, which must itself be able to coerce the pointee.

If I understand what you're saying, that's essentially what you need the compiler to do there.

[CAD97]

@aapoalas

The only addition I think is a blanket impl on Deref!

impl<T: Reborrow + Deref> CoerceShared for T where T::Target: Copy {
    type Output = T::Target;
}

What this is implying is that your MyMut derefs to MyRef, i.e. fn(&MyMut) -> &MyRef. I don't think that's what you're wanting, and in the case that you have that deref, you don't need a separate way to do the coercion as well. It can be a way to handle things today, implementing fn deref as NonNull::<MyMut>::from(self).cast::<MyRef>().as_ref() (or equivalent), but this is more of a hack than anything else (to allow methods of MyRef to be called on a receiver of type MyMut), and means you can't DerefMut usefully; you're abusing Deref as an implicit receiver coercion when you aren't actually a "container" type^[1].

Do note that my proposed blanket impl from Copy to CoerceShared isn't strictly necessary for any reason; it exists to put a limit on the coercion space, by ensuring that the CoerceShared::Output of any value cannot be further coerced (as it coerces into itself). This is neater than CoerceShared: !Copy or CoerceShared::Output: !Copy (negative bounds are not great in general) and perfectly self consistent, although the mutual exclusion with Copy would also be self consistent.

@aapoalas

indeed Deref::Target and CoerceShared::Output differing from one another should definitely be an error

You've interpreted this a bit incorrectly, I believe. To illustrate:

Self	Deref::Target	CoerceShared::Output
&T	T	&T
&mut T	T	&T

The equivalence we want for method lookup reason is actually that <Self as Deref>::Target is the same as <<Self as CoerceShared>::Output as Deref>::Target.

Then lookup order for a method f on a receiver place of type T goes something like:

• Let BasicLookup(T, f) be:
  • If T implements an intrinsic method f(self: T), call that. Else:
  • If T implements a trait method f(self: T), call that. Else:
  • Fail.
• Let ReceiverLookup(T, f) be:
  • If T implements Receiver^[2]:
    • If T::Target implements an intrinsic method f(self: T), call that. Else:
    • If T::Target implements a trait method f(self: T), call that. Else:
  • Fail.
• Let Lookup(T, f) be:
  • Do BasicLookup(T, f). Else:
  • Do ReceiverLookup(T, f). Else:
  • Do ReceiverLookup(&T, f). Else:
  • Do ReceiverLookup(&mut T, f). Else:
  • (new!) If T implements CoerceShared:
    • Do ReceiverLookup(T::Output, f). Else
  • If T implements Deref:
    • Do Lookup(P::Target). Else:
  • Fail.
• Do Lookup(T, f). Else fail.

For name lookup to work reasonably, we want to restrict this algorithm to only tail recursion. Luckily, only doing ReceiverLookup for method lookup on CoerceShared::Output seems reasonable (when we have the restriction that the receiver target of T and T::Output must match). Compare calling a method on a place with type &mut T, which does not see any self methods implemented for &T, only &self methods implemented for T. If my quick tests were accurate.

Is it reasonable to have CoerceShared require Receiver? It should be; if you have a mut/ref pair, you almost certainly want to let the ref functionality be shared instead of duplicated. But it might require extern type to become a thing before that becomes practical. (But there's a new push in that direction!)

---

1. Note I didn't say "pointer" type. Deref originally said it was for pointer-like types, but the use for things like ManuallyDrop or LazyCell is also perfectly reasonable; it's valid when you want a value of type &Container to be usable as if it were &Target. But if you want a place of type Mut to be usable as a value of type Ref, you want CoerceShared, as I understand it. ↩︎

2. Blanket implemented for all impl Deref. ↩︎

[aapoalas]

@CAD97

You can't borrow a shared lifetime exclusively. That's fundamentally not a thing.

When you reborrow Scope<'a, 'b> where both lifetimes are covariant, you get Scope<'m, 'n> where 'a: 'm, 'b: 'n. What exactly that means depends on what set of loans those lifetimes represent. If 'a captures any exclusive loans, then using any type mentioning 'a (or otherwise invalidating 'a) must come after invalidating 'm. The same goes for 'b and 'n. But if the lifetime only captures shared loans, then the parent lifetime can be used without invalidation of the child lifetime.

This, exactly, already happens whenever you turn a place with type Scope<'a, 'b> into a value (rather than borrow it). What's missing is the ability for the compiler to return access to the source place after the child lifetimes expire, as the only option for user defined types is move or independent copy.

Consider also that fn(&mut T) -> Type<'_> and fn(&T) -> Type<'_> produce the "same" type with the "same" lifetime position, but each captures a distinct loan set, so the former prevents access to the input reference, but the latter doesn't.

But how does the compiler know which lifetime is an which isn't based on a shared loan?

Consider that function but with a custom reference type parameter with multiple lifetimes, changing the return lifetime instead of changing &mut and &: fn(Source<'a, 'b>) -> Type<'a> and fn(Source<'a, 'b>) -> Type<'b>. Which signature captures exclusive loans? Which one captures shared loans? Of course there's no way to tell, not necessarily even if you look inside Type since it's probably just using PhantomData fields for the lifetimes.

Now imagine that a function call fn(Source<'a, 'b>) reborrows Source as exclusive, and calls the function that returns Type: in which version does the original Source stay disabled for reads and writes (captures exclusive) after the inner call finishes, and in which version does it only stay disabled for writes (captures shared)?

fn outer(source: Source) {
  let result: Type = fn(source); // source is reborrowed as exclusive
  other_fn(source); // source is reborrowed as exclusive
  result.use();
}

How does the compiler know if this is a compilation error, or if result is actually capturing only a shared reference from within Source and thus retaining it is perfectly fine? It of course cannot know: currently the answer isn't local since it depends on the source loans of these lifetimes and those are not within the function.

Maybe it's worth noting how my GcScope<'a, 'b>'s reborrowing is implemented:

fn nogc(self: &'outer mut GcScope<'a, 'b>) -> GcScope<'outer, 'b>

ie. the second lifetime is not touched by reborrowing, and it doesn't need to be since it's a shared reference lifetime: all who capture it can and should capture its "full lifetime" so to say, instead of narrowing it.

@CAD97

See how derive(CoercePointee) solves that exact thing. You mark the #[pointee] type parameter and are required to only have a single field mention that parameter, which must itself be able to coerce the pointee.

If I understand what you're saying, that's essentially what you need the compiler to do there.

I'm not quite sure, but I don't think this is not the same thing. CoercePointee requires the repr(transparent) requirement, whereas I'm mostly thinking about reborrowable multi-field cases. I'm thinking of the issue of reborrowing MyMut as shared MyRef in such a way that they are allowed to have different memory layouts but if you can always find a MyRef "inside" of a MyMut then that could be used as a safe "step 0" for shared reborrowing.

eg. Imagine both MyMut and MyRef contain a single PhantomData and two u32s. Implementing CoerceShared from MyMut to MyRef is basically trivially safe here but how can the compiler know that? One thing would be if Deref of MyMut showed that you can take a &MyMut and turn that into a &MyRef, with the code for doing so being just the identity function. Now the compiler can go, "okay, I have a MyMut, turn that into &MyMut which turns into &MyRef which I can Copy out. QED."

What if MyRef only contains a single u32? Now the Deref is either the identity function or a 4 byte offset, but either way having it defined works for the compiler to know how it can turn MyMut into a MyRef since it can deref from &MyMut to &MyRef and MyRef is Copy.

[aapoalas]

@CAD97

What this is implying is that your MyMut derefs to MyRef, i.e. fn(&MyMut) -> &MyRef. I don't think that's what you're wanting, and in the case that you have that deref, you don't need a separate way to do the coercion as well. It can be a way to handle things today, implementing fn deref as NonNull::<MyMut>::from(self).cast::<MyRef>().as_ref() (or equivalent), but this is more of a hack than anything else (to allow methods of MyRef to be called on a receiver of type MyMut), and means you can't DerefMut usefully; you're abusing Deref as an implicit receiver coercion when you aren't actually a "container" type^[1].

Hmm, yeah... You're probably right. I got too enthused about getting a way to get the compiler a way to automatically derive shared reborrowing and get method lookup at the same time. I was thinking about this from the point of view of Mut and Ref themselves being the important types (as they are for me; my GcScope doesn't have any type parameters beyond the two lifetimes), and disregarding the possible Mut<'_, T> container usage. Though, I was somewhat thinking that if you have a custom reference container type that can Deref(Mut) into its contained type, then why aren't you using the actual reference directly? But that's of course naive on my part, there are many reasons to do this and my presumption on this not being the case is not a great look.

@CAD97

You've interpreted this a bit incorrectly, I believe. To illustrate:

Self	Deref::Target	CoerceShared::Output
&T	T	&T
&mut T	T	&T

The equivalence we want for method lookup reason is actually that <Self as Deref>::Target is the same as <<Self as CoerceShared>::Output as Deref>::Target.

Then lookup order for a method f on a receiver place of type T goes something like:

• Let BasicLookup(T, f) be:
  • If T implements an intrinsic method f(self: T), call that. Else:
  • If T implements a trait method f(self: T), call that. Else:
  • Fail.
• Let ReceiverLookup(T, f) be:
  • If T implements Receiver^[2]:
    • If T::Target implements an intrinsic method f(self: T), call that. Else:
    • If T::Target implements a trait method f(self: T), call that. Else:
  • Fail.
• Let Lookup(T, f) be:
  • Do BasicLookup(T, f). Else:
  • Do ReceiverLookup(T, f). Else:
  • Do ReceiverLookup(&T, f). Else:
  • Do ReceiverLookup(&mut T, f). Else:
  • (new!) If T implements CoerceShared:
    • Do ReceiverLookup(T::Output, f). Else
  • If T implements Deref:
    • Do Lookup(P::Target). Else:
  • Fail.
• Do Lookup(T, f). Else fail.

For name lookup to work reasonably, we want to restrict this algorithm to only tail recursion. Luckily, only doing ReceiverLookup for method lookup on CoerceShared::Output seems reasonable (when we have the restriction that the receiver target of T and T::Output must match). Compare calling a method on a place with type &mut T, which does not see any self methods implemented for &T, only &self methods implemented for T. If my quick tests were accurate.

Is it reasonable to have CoerceShared require Receiver? It should be; if you have a mut/ref pair, you almost certainly want to let the ref functionality be shared instead of duplicated. But it might require extern type to become a thing before that becomes practical. (But there's a new push in that direction!)

Thank you, this seems like a very reasonable thing and having CoerceShared require (or blanket impl?) Receiver definitely makes sense with the added receiver lookup step (or seems so to me with your excellent explanation <3).

I was somehow thinking about this in the sense of trying to unify &mut T, &T, Mut<'_, T>, and Ref<'_, T>, such that (theoretically at least) &mut T would just be another Mut<'_, T> that implements CoerceShared into &T. But my type system chops (which don't exist, to be exact) are failing me and I can't really carry that thought through right now

---

1. Note I didn't say "pointer" type. Deref originally said it was for pointer-like types, but the use for things like ManuallyDrop or LazyCell is also perfectly reasonable; it's valid when you want a value of type &Container to be usable as if it were &Target. But if you want a place of type Mut to be usable as a value of type Ref, you want CoerceShared, as I understand it. ↩︎

2. Blanket implemented for all impl Deref. ↩︎

[CAD97]

@aapoalas

How does the compiler know if this is a compilation error, or if result is actually capturing only a shared reference from within Source and thus retaining it is perfectly fine? It of course cannot know: currently the answer isn't local since it depends on the source loans of these lifetimes and those are not within the function.

This isn't quite right, I don't think. The subtlety will absolutely mean that implementing Reborrow with complicated type invariants will be tricky, but this compiles:

fn outer(mut source: Source<'_, '_>) {
    let x: Ref<'_> = f1(re(&mut source));
    let _: () = f2(re(&mut source));
    let _: () = f3(x);
}

given this context:

type Ref<'a> = &'a ();
type Mut<'a> = &'a mut ();
type Source<'a, 'b> = (Mut<'a>, Ref<'b>);
fn re<'a: 'n, 'b: 'm, 'n, 'm>(_: &'n mut Source<'a, 'b>) -> Source<'n, 'm>;
fn f1<'b>(_: Source<'_, 'b>) -> Ref<'b>;
fn f2(_: Source<'_, '_>);
fn f3(_: Ref<'_>);

and this isn't reliant on the fact that the fields that carry the lifetimes are public, either; it works just the same with struct Source<'a, 'b>(Mut<'a>, Ref<'b>) in a module so that only fn re (reborrow emulation) can see the fields. And to be clear, if you switch which lifetime is borrowed by f1, it stops compiling.

This comes out of the fact that fn re is declared as essentially fn(&'n mut Source) -> Source<'n, '_>. But I didn't make any decisions here; the compiler told me exactly what was needed. I originally wrote

fn re<'a: 'n, 'b: 'm, 'n, 'm>(scope: &'_ mut Source<'a, 'b>) -> Source<'n, 'm> {
    (scope.0, scope.1)
}

and the compiler told me

error[E0621]: explicit lifetime required in the type of `scope`
 --> src/lib.rs:7:5
  |
6 | fn re<'a: 'n, 'b: 'm, 'n, 'm>(scope: &'_ mut Source<'a, 'b>) -> Source<'n, 'm> {
  |                                      ---------------------- help: add explicit lifetime `'n` to the type of `scope`: `&'n mut (&'a mut (), &'b ())`
7 |     (scope.0, scope.1)
  |     ^^^^^^^^^^^^^^^^^^ lifetime `'n` required

I'm implicitly expecting the compiler will do exactly this analysis for reborrowing user types; a reborrow is essentially Self { ..self }.

But perhaps the issue is that if we instead store the lifetimes in PhantomData instead, the compiler no longer sees this relation, as PhantomData is Copy where you want only Reborrow for 'a. (If we make a type with only Copy fields inferred-Reborrow, it would become effectively Copy in its usage.)

I had been overlooking this detail. So we need to either block access to the source until all reborrowed lifetimes expire^[1], or provide some way to mark specific lifetimes as mutably reborrowed even if that isn't structurally necessary.

If we follow the (realistically compiler internal only) example of the current CoerceUnsize, it would be impl<'a, 'b: 'a, 'c> Reborrow<Source<'a, 'c>> for Source<'b, 'c>>. The unsizing machinery shows that the compiler could check such, but the choice to expose derive(CoercePointee) instead shows that, while it is functional, this isn't a very nice API.

---

Oh, and just to note: fn re can be simplified to fn re<'a, 'b>(scope: &'a mut Source<'_, 'b>) -> Source<'a, 'b> for effectively the same behavior. This usage is even fine with type Source<'a, 'b> = &'a mut &'b () and that gives it sufficient auto reborrows, even, because 'b isn't required to be covariant here. [playground]

---

Aside: there's definitely more magic going on than I have a full grasp around. fn f(x: &'a &'b i32) -> &'b i32 can be implemented as &**x but not as Deref::deref(x). The best I can tell, the compiler treats &* on a place of type &_ as a no-op and ignores it, despite the fact that this is only fully true for values in the absence of this special case. But that's not quite right, either; the debug MIR for &**x is deref_copy (*_1), but &*{*x} is copy (*_1), but just *x is deref_copy (*_1) again. I think I need to look at MIR even earlier (but I'm just using the playground, so I can't use -Zdump-mir easily).

---

1. Roughly,

   pub fn re<'a, 'b, 'n, 'm, 'o>(source: &'o mut Source<'a, 'b>) -> Source<'n, 'm>
   where
       'a: 'n,
       'b: 'm,
       'o: 'n,
       'o: 'm,
   {
       (source.0, source.1)
   }
   

   ↩︎

[aapoalas]

Regarding this function signature:

fn re<'a: 'n, 'b: 'm, 'n, 'm>(_: &'n mut Source<'a, 'b>) -> Source<'n, 'm>;

as you noted, this already works currently but note that this is not true reborrowing. This is the fascimile that we have currently in userland. If we return x from fn outer then it will work, presumably because the compiler actually goes through the "shared reborrowed as shared" path and just allows the 'm lifetime to stay equal to the original 'b? (I'm not even sure if there is lifetime narrowing when passing/reborrowing shared references; do they need that in the first place? I've been somewhat operating under the assumption that such a thing doesn't actually need to and hence doesn't exist.)

But if we then make f2 return a Mut<'n> and try to return that from fn outer as Mut<'a>, the compiler will not allow this to pass: the &'n mut Source that is used to create Mut<'n> is a local variable and the lifetime explicitly says that Mut<'n> captures it, ie. it captures something on the current function's stack. Obviously that cannot escape from the function, so this doesn't compile. We both know that this isn't actually what we're doing here, we just want to narrow from Mut<'a> to Mut<'n> where 'a: 'n but the compiler of course doesn't understand that since we have no way to explain it to it.

Furthermore, in a sense the Source type you're describing is a variant of "reference wrappers"; they hold something truly reborrowable inside already (references), and thus pondering on that is kind of trying to figure out the answer to reborrowing Option<&mut T> or Pin<&mut T>. I'm focusing more on the case of PhantomData reborrowing. In a way I think this is kind of the question of "how to describe reborrowing of &mut T without using reference reborrowing?", or an "axiomatic" construction of reborrowing in general.

Regarding storing lifetimes in PhantomData: if narrowing of shared lifetimes is unnecessary (as you noted in the footnote with fn re<'a, 'b>(scope: &'a mut Source<'_, 'b>) -> Source<'a, 'b>; this is effectively the same behaviour), then one thing could be for the compiler to skip lifetime narrowing on Reborrow: Copy fields, and only perform it on Reborrow: !Copy fields. Then the only thing we'd need is a marker type:

// in std::marker

pub struct PhantomExclusive<'a>; // or maybe "PhantomReborrow"
pub struct PhantomShared<'a>;

impl !Copy for PhantomExclusive<'_> // maybe !Clone?

impl<'a> Reborrow for PhantomExclusive<'a> {}

impl<'a> CoerceShared for PhantomExclusive<'a> {
  type Output = PhantomShared<'a>;
}

Now a custom exclusive reference wrapper would be a struct that contains a field containing the PhantomExclusive type inside it, and then optionally other fields that can be Copy or otherwise Reborrow. (I added the lifetime to PhantomExclusive since it seemed apt, but it could be outside of it just as well.) A custom shared reference wrapper is then one that only contains Copy fields.

Now the compiler could use this marker type to sniff out which lifetimes are exclusive and which are shared. This would also (aspirationally) make it possible to define &mut and &:

// in std::reference

struct Mut<'a, T> {
  ptr: NonNull<T>,
  noalias: PhantomNoAlias, // somehow say that ptr is not aliased
  lifetime: PhantomExclusive<'a>,
}

struct Ref<'a, T> {
  ptr: NonNull<T>,
  lifetime: PhantomShared<'a>,
}

Now CoerceShared from Mut to Ref could be automatically checked field-wise by checking that ptr field types are match and are Reborrow: Copy (no exclusivity), and lifetime field types are T and <T as Reborrow>::Output respectively.

[aapoalas]

I've now updated both the opening message text and my repository to use @CAD97 's recursive formulation, and added (hopefully) more proper guide and reference level explanations.

[PoignardAzur]

This looks good! I mentioned reborrows at RustWeek and Josh Triplett was pretty enthusiastic about the prospect of eventually adding them, so it's nice to see movement on this.

I especially like the CoerceShared trait.

I was a little surprised at first Reborrow didn't have an associated type, but on second thought it makes sense, because the type it's going to be reborrowed to is going to depend on the call site, and enforcing that this type is a subtype of Self will need to be handled by compiler magic anyway.

[PoignardAzur]

My one complaint would be that the RFC feels a little hard to read right now. It feels like a lot of it could be made more concise and readable with some editing.

[aapoalas]

I definitely agree. I've only really had time to add more context and thoughts as I've had them, and haven't yet gotten around to trimming off all the excess. That will definitely need to be done, though.