Can `Pin` own its referent instead of borrowing mutably?

language design
15

Inspired by this thread and the recent DynSized RFC, I had an idea for how to replace Pin with a forwards-compatible, library-only subset of !Move. Which is actually just !DynSized.

In ‘short’:

  • Pin<'a, T> is replaced by &'a mut T. So Future, Generator, etc. can just take &mut self again! Here I’ll talk about Generator for convenience, but the same applies to Future.
  • To enforce immovability, users never get access to owned generators directly; thus, instead of generator functions returning impl Generator, they would return something like impl Anchor<Inner=impl Generator>, for some trait
    trait Anchor {
    type Inner: ?Sized;
    // PinBox and friends would use this method to implement DerefMut;
    // it's unsafe because you should only call it if you can guarantee
    // that `self` will never move again.
    unsafe fn get_mut(&mut self) -> &mut Self::Inner;
}
  • Okay, the hard part. Why doesn’t the above ‘just work’?
    • Because of mem::swap and friends. Given &mut MyGenerator you can move it or (in some cases) extract an owned MyGenerator, which breaks the assumption of immovability.
  • So what if we mark MyGenerator as !Sized? (Notwithstanding the semantic strangeness of making something !Sized that doesn’t actually have an unknown size.) That would block mem::swap and similar.
    • But there’s one dangerous function that doesn’t have a Sized bound: mem::size_of_val. In particular, the Box-to-Rc conversion works even for unsized types and assumes it’s kosher to move them by memcpying mem::size_of_val(foo) bytes.
    • Although you couldn’t actually use that conversion to break the above scheme, since you’d never be allowed to get a Box<MyGenerator>, its presence in the standard library is a license for unsafe code to use the same technique in broader scenarios.
  • But this is the same problem meant to be solved by !DynSized!
    • The above-linked RFC proposes having size_of_val either panic or return 0 for a !DynSized type, combined with a lint; an alternative mentioned is to make DynSized a new default bound and add ?DynSized.
    • Meant to be used with extern type, i.e. FFI opaque pointers, which itself is already accepted and implemented (but not stable).
    • But it would work just as well for immovable types. The desired semantics are effectively the same at least from a generic code perspective.
  • But even ignoring how crazy that sounds, how is it a library-only solution if DynSized doesn’t even exist yet?

Well, for now we can simulate it with a silly hack. Generator functions would expand to something like:

// The anchor contains the actual state...
struct MyGeneratorAnchor {
    // local variables go here...
    x: i32,
}

// The generator type is just an extern type!
// In other words, &MyGenerator is an 'opaque pointer' that secretly points
// to MyGeneratorAnchor.
extern { type MyGenerator; }

impl Anchor for MyGeneratorAnchor {
    type Inner = MyGenerator;
    unsafe fn get_mut(&mut self) -> &mut Self::Inner {
        // create the opaque pointer by casting to &mut MyGenerator:
        &mut *(self as *mut MyGeneratorAnchor as *mut MyGenerator)
    }
}

impl Generator for MyGenerator {
    fn resume(&mut self) -> whatever {
        // undo the above cast:
        let anchor: &mut MyGeneratorAnchor = unsafe { &mut *(self as *mut MyGenerator as *mut MyGeneratorAnchor) };
        // … now we can use local variables …
    }
}

So, how does this solve the problem?

  • MyGenerator is !Sized (extern type already has that effect), so anything that tries to use the standard mem::swap, mem::size_of, etc. just won’t compile.
  • mem::size_of_val(foo: &MyGenerator) will work but return 0 (because it’s an extern type). So if some tricky unsafe code tries to, say, swap two &mut MyGenerator instances by memcpying bytes around, that’ll just silently do nothing, leaving the generators’ actual data intact. That’s not great, in the sense that it’s a surprising result, but it’s not unsound either. And again, this is the exact same problem that applies to ‘real’ FFI opaque pointers.
    • …There might be some even weirder things unsafe code might try to do that would be unsound – though it’s hard to say it’d be justified in doing so – but that would be equally unsound with extern type used for FFI! (For example, a version of take_mut that takes ?Sized and returns the existing object copied to a box.)
  • In the hopefully near future, the DynSized RFC or something similar will land, providing for a lint or error when someone tries to use size_of_val here.
  • In the meantime, since essentially no generic code actually does that, you’ll be able to use the generic ecosystem as usual with &MyGenerator and &mut MyGenerator – with no need for everything to add an extra variant for Pin. ?Sized is enough.

Thoughts?

2 Likes