Can `Pin` own its referent instead of borrowing mutably?

[comex]

@steven099

Viewing Pin<T> as a wrapper for &mut T for weird T begs the question somewhat, since you’re arguing for it to be equivalent to &mut T where T: !DynSized. Also, to the extent that this describes the implementation, you’re violating the abstraction barrier. Pin<'a, T> can be viewed in its own right as a unique reference type with lifetime 'a which, for T: !Unpin, can only exist where &mut / move access has been permanently disabled for the referent.

Fair enough. But – first of all, I don't think this question really affects any of the points I've made. I wrote:

After all, just as Pin<'a, T> is a wrapper for &'a mut T for weird T, FFI users could make MyFFIReference<'a> wrapper structs; but extern type provides extra sugar to allow native references to be used safely, a goal that I think applies here as well.

I guess that if Pin is seen as a fundamental reference type, the guidance could be that instead of making your own MyFFIReference<'a> wrapper, you should design your APIs around Pin<'a, MyFFIType>. But then what do you use for non-unique references? &MyFFIType doesn't work because of the same issue with size_of_val. You still need DynSized. On the other hand, if you purely used, e.g., MyFFIReference<'a> and MyFFIReferenceMut<'a>, then you wouldn't need either DynSized or extern type, but the API would be less ergonomic, which is the same issue that applies to Pin.

In any case, my problem with viewing Pin as a fundamental reference type is that at least for general self-referential structs, there are also use cases for reference types that correspond to & and, potentially, &move (the original proposal in this thread), while guaranteeing immovability. So we'd end up with 6 reference types instead of 3 (or without &move, 4 instead of 2).

For example, this is unsound using regular &:

struct S {
    foo: [i32; 32],
    // points to one of foo's elements:
    bar: Cell<Option<&'foo i32>>, 
}
impl S {
    fn set_bar(&self) {
        self.bar.set(Some(&self.foo[0]));
    }
}

But it would be sound with an immovable version of &.

(&Pin<T> could work as a substitute, but has limitations due to being a double pointer, e.g. you can't go from &Pin<Struct> to &Pin<SomeField>.)

[edit: See also @RalfJung's recent post Safe Intrusive Collections with Pinning, which explores a somewhat different direction than what I'm proposing, but also concludes that an immutable pin is useful. I think the invariant he wants can also be achieved with my Pin-less approach.]

Anyway...

You can’t reborrow it for a shorter lifetime;

https://doc.rust-lang.org/nightly/std/mem/struct.Pin.html#method.borrow

I missed that, but let's correct the point to: "you can't implicitly reborrow it for a shorter lifetime".

In principle, yes, but extern types are a bit of a necessary evil unless/until the soundness issues can be fully addressed. The opportunity exists to have immovable types enter the world entirely absent these issues.

I think that's short-term thinking. To the extent that soundness issues exist, they're an urgent problem and need to be solved before extern type can be stabilized. (Or else there's not much benefit to having it when unit structs already satisfy the unsound-but-mostly-works use case.) However, I don't think they're a big deal; in particular, I'm not sure whether any code actually exists that uses size_of_val to copy bytes from a non-owned type. And the DynSized RFC proposes having size_of_val panic (instead of returning 0), which will eliminate any such unsoundness, at the cost of surprising behavior.

Box-to-Rc comes to mind. I’m pretty sure you’d end up with an uninitialized T at the end of it. I’m not convinced by the argument that immovable types will always be sufficiently abstract that you can’t put them in a Box. Maybe you can come up with a scheme that works for generators, but that’s far short of a generalized feature, and doesn’t address what external library code might (validly?) do.

Regarding Box-to-Rc specifically: in the short term, exposing a Box<Immovable> to safe code would be unsound because you could move out of it.

In the medium term, having size_of_val panic fully removes the potential for unsoundness, again at the cost of surprising behavior.

In the long term, the goal is to have the standard library include appropriate DynSized bounds so the compiler can reject such code at compile time. The DynSized RFC proposes one way of doing that while minimizing disruption.

Actually, I'd only skimmed that RFC, and it turns out I misunderstood what it was proposing . Now that I understand, I like it much better! And think it's clearly better than !DynSized. I thought it was proposing a lint at monomorphization time if size_of_val etc. is instantiated with !DynSized types. But it's actually proposing that for each caller, the compiler try to prove, in a generic context, that T: DynSized holds, and lint if it can't. So to avoid the lint, callers must change their bounds to add either T: DynSized or #[assume_dyn_sized], either of which propagates the requirement outward. Thus, as long as your code compiles without producing the lint, you have a strong guarantee it's fully DynSized-safe, even in the fact of clients instantiating your generic code with arbitrary parameters.

You probably already knew that, but I didn't, and after thinking about it, I'm now much more confident that DynSized can be adopted quickly and painlessly. I also commented on the RFC with a suggestion: instead of adding a special-case #[assume_dyn_sized], we can get the same effect using specialization, if the compiler is changed to support #[deprecated] on impls.

Moving on:

If the idea is for this proposal to be fully backward-compatible with the existing Pin design, then this doesn’t work.

No, it's not meant to be backwards-compatible with Pin. Even if it's a bit late, there's still some time to kill Pin before it's stabilized.

The current design of Pin currently has that for the &T case. But yeah, having it just work for &mut T where T: ?Sized is nice, though I’m not sure how common that is compared to &T where T: ?Sized. I’m not entirely convinced about the need for distinguishing whether the referent is movable vs immovable for &T.

Above I showed a use case for immovable &T – basically any use of interior mutability with self-references. The current design… oh, I missed the unconditional Deref impl on Pin even for T: !Unpin. So you get to work with native &T, at the cost that immovable types based on Pin can never have interior mutability. Not great, IMO...