A backward compatible `Move` trait to fix `Pin`

language design
1

Posting to internals forum on recommendation from quindot here.

I posted about a question about ?Move backwards compatibility on Reddit and didn't get a good answer. So I'm going to apply Cunningham's_Law and assume I'm right.

I'll assume P: !Move + !Unpin and U: Move + Unpin for the remainder of this post.

Summary

?Move is backwards compatible just like Pin is. The definition used by [1] and [2] both work for this explanation, but I'll go with: T: !Move means "safe code may never move T and unsafe code may only move T if the Move type's address has not been witnessed.

The short description is:

  1. Replace all instances of Pin<Ptr> with Ptr and !Move trait as needed. Replace all instances with -> P which is later pinned to Pin<&mut P with -> U where U has a method later used to initialize P with an out parameter (e.g. into_future<P: ?Move + Future>(self) -> super P [3]).
  2. Add two new Deref traits corresponding to Deref and DerefMut where Target: ?Move.

Not breaking

[2:1] argues that Move is unusable because of backward incompatibility. Most examples given don't actually present issues though.

Traits which can present a problem must have the following characteristics:

  • Returns a type which mentions an associated type without moving the associated type (like &P or &mut P ).
    • If the associated type is moved by value, it can't be !Move anyway.
    • This eliminates the arithmetic operators which return associated type by value.
    • This eliminates Iterator which return associated type by value.
  • The type doesn't interact with or use Pin.
    • This proposal replaces Pin with the Move trait, so traits which explicitly rely on Pin will be substituted with equivalent ?Move where needed.
    • This eliminates the normal Future and IntoFuture types.
  • The trait is implemented on Pin currently.
    • Traits like Index{,Mut} aren't implemented for Pin showing a clear lack of need for, or inability to use, stable addresses.
    • As{Mut,Ref} are not implemented for Pin and instead are inherent impls on Pin. Normal reference reborrowing on &T and &mut T (with the addition of the two new traits) can replace these.
    • This eliminates all traits that aren't implemented for Pin<Ptr> where Ptr's Target: !Unpin leaving Deref.

To fix the inability to use Deref and DerefMut, create two new traits DerefNotMove and DerefMutNotMove (bikeshed) which are the same as Deref and DerefMut but their associated type is ?Move. These new traits will be added to deref coercion.

  • The traits could be made so that implementing Deref prevents DerefNotMove and DerefNotMove, is implemented for all Deref, and only the Deref{,Mut} is special to the compiler.
  • Or Deref and DerefMut could become aliases to DerefNotMove<Target: Move> and a limited specialization would allow implementing both depending on T: Move/T: !Move.
  • Or they could simply be additional special traits.

Comparison to Pin

Any api which wishes to take advantage of the pinned typestate and stable addresses must either accept the parameter as Pin or bound by ?Move, but both require a change. For example if the api takes an argument &mut T it doesn't support a T with a stable address and can't rely on T having a stable address. If T: !Unpin is pinned then it can't be used in api's that require &mut T. This is the same as with Move.

In the world of Pin if T is the type you return from a function, you need a new type Pin<&mut T> with T: Future to poll it. In the world of Move if T is the type you return from a function, you need a new type P: Future to poll it. The difference is how T is converted to the new type. Pin requires a new_unchecked after custom setup code for T. Move requires putting that custom code in a into_future<T: IntoFuture, P: ?Move + Future>(T) -> super P.

Using Index as an example, to use stable addresses with IndexMut requires IndexMut<Output = Pin<&mut P>> and method signature becomes index(&self, index: Idx) -> &mut Pin<&mut P>. To do the same with Index<Output = &mut P> is also allowed (since &mut P: Move) and the signature becomes index(&self, index: Idx) -> &mut &mut P which is the same thing as the Pin version.

Function traits are also the same. You can't return P from a function without using super which changes the function signature. This is the same as Pin<&mut T>. You can't return, by regular move, a type T after its pinned. To return a &P or &mut P is not problem since &P: Move and &mut P: Move, just like Pin<&P> and Pin<&mut P>.

New traits

If Iterator took self by Pin instead, this would require a new trait. This is what's happening with the AsyncIterator/Stream proposal. If Iterator bounded self with ?Move this would require a new trait. Both approaches require a new trait.

A similar argument can be made for every new trait which takes a Pin.

Migrating

The following implementations can ease transitioning to Move based APIs.

    // It is safe to pin or remove a pin on an immovable type. The type can't move so the `Pin` does nothing.
    impl<Ptr> Pin<Ptr>
    where
        Ptr: core::ops::Deref,
        <Ptr as DerefNotMove>::Target: !Move,
    {
        /// Constructs a new `Pin<Ptr>` around a pointer to some data of a type that doesn't implement `Move`.
        fn new_immovable(ptr: Ptr) -> Pin<Ptr> {
            unsafe { Pin::new_unchecked(ptr) }
        }
        /// Unwraps this Pin<Ptr>, returning the underlying pointer.
        /// Doing this operation safely requires that the data pointed at by this pinning pointer doesn't implement `Move` so that we can ignore the pinning invariants when unwrapping it.
        fn into_inner_immovable(pin: Pin<Ptr>) -> Ptr {
            unsafe { Pin::into_inner_unchecked(pin) }
        }
    }
    #[repr(transparent)]
    struct PinFutureFromMoveFuture<T: !Move>(T);
    impl<T: core::mov::Future> core::future::Future for PinFutureFromMoveFuture<T> {
        type Output = T::Output;

        fn poll(
            self: Pin<&mut Self>,
            cx: &mut Context<'_>,
        ) -> Poll<Self::Output> {
            // Safety: T: !Move so `poll` can't move out of the `Pin`.
            let me = unsafe { self.get_unchecked_mut() };
            me.0.poll(cx)
        }
    }
    #[repr(transparent)]
    struct MoveFutureFromPinFuture<'a, T>(Pin<&'a mut T>);
    impl<'a, T: core::future::Future> core::mov::Future for MoveFutureFromPinFuture<T> {
        type Output = T::Output;

        // Required method
        fn poll(
            self: &mut Self,
            cx: &mut Context<'_>
        ) -> Poll<Self::Output> {
            self.0.as_mut().poll(cx)
        }
    }

async fn desugaring

Like in [1:1], an async fn returns an IntoFuture<IntoFuture: ?Move> instead of a Future. The IntoFuture itself is Move. .await first calls into_future and then polls.

Stable rust out parameters.

Also there's a way to implement a usable desugaring of -> super on stable Rust today. [3:1] describes most of the process. The bit that's missing is that .assume_init() which moves the value, is not necessary (though it would be much better). Instead use an owned reference (&own) similar to the Stackbox type and cast the ptr/reference as needed. After all, what else can an owned !Move value be used for beyond delayed Drop or reference? This gets even messier (but still possible?) with enums since you can't easily get the address of the variant's fields while the out pointer is uninitialized.

Some code related to this idea is here in case it helps understand what I'm saying.

The thing I would like feedback on the most is if this idea is not crazy, and whether ?Move is in fact not necessarily backward incompatible.

  1. Ergonomic Self-Referential Types for Rust ↩︎ ↩︎

  2. Pin ↩︎ ↩︎

  3. in-place construction seems surprisingly simple? ↩︎ ↩︎

1 Like
2

How is this not breaking? There is code currently assuming that async fns return movable values.

Moreover IntoFuture satisfies this point of yours:

It returns its IntoFuture associated type by value, hence that cannot be made ?Move according to your observations.

You can create yet another trait, but this just decreases the chances to get it accepted and doesn't really help with the incompatibility due to changing what async fns return.

2 Likes
3
  1. I forgot to ask, Would this have been breaking if it was introduced instead of Pin?
  2. Can editions not change the meaning of syntax? What prevents a future edition from making async work with Move (or adding a modifier to async fns)? Code compiled on previous editions would assume the result of async is movable and later, not.
  3. async fns still return movable values even though its a different trait, (mov::IntoFuture instead of future:Future).
4

Code from different editions can be freely mixed. Due to macros, you can have each line of code have a different edition.

Editions are only shallow tweaks, almost just syntax sugar. You need to be able to explain how code from all editions can be mapped to a single common representation that works with all of them at once. This means you can't have the same trait that has different meanings per edition. You could make each edition have its own trait, but then they wouldn't be interoperable after desugaring to the common edition.

1 Like
5

Yes, there would be an alternate IntoFuture and Future trait which would look like:

pub trait Future
    where
        Self: ?Move,
{
    type Output;

    fn poll(self: &mut Self, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

pub trait IntoFuture {
    type Output;
    type IntoFuture: Future<Output = Self::Output>;

    fn into_future(self) -> super Self::IntoFuture; // or equivalent emplacement mechanism
}

but this was not an issue before Pin was introduced and doesn't answer the question of would this design have been backward compatible if it was chosen instead of Move.

The two versions are interoperable. You can write in one imagined "unified" form as:

async_pin fn f1() -> T1 {..} translates to fn f1() -> impl core::future::Future
async_mov fn f2() -> T2 {..} translates to fn f2() -> impl core::mov::IntoFuture

and use both like

let fut1 = MoveFutureFromPinFuture(pin!(f1()));
let fut2 = f2().into_future();
fut1.await;
fut2.await;

Or the reverse with the PinFutureFromMoveFuture;

This change is only changing syntax sugar per edition: changing the de-sugaring of async and .await.