Can `Pin` own its referent instead of borrowing mutably?

[steven099]

Unless calling that conversion function sets a bit flag preventing future unpinned accesses. But yeah, otherwise it’s incompatible. Of course, pretty much all interior mappings (including field access) are unsafe, so if the conversion were unsafe, a pinned type could expose a &RefCell<Pinned<T>> and enforce the contract ("no &RefCell<T>") through encapsulation. That’s one thing that’s always going to be a challenge about pinning under either approach: just how much pinned types are responsible for maintaining soundness in their public APIs.

[withoutboats]

The conflation with “unsized” doesn’t actually make sense. Something without a dynamically known size can’t be deallocated, for example. This would imply that we leak all generators, and there’s no reason for it.

[steven099]

(Points to Move proposal upthread.)

I’m not sure, but wouldn’t the allocator know the real size of the value?

[RalfJung]

Unless calling that conversion function sets a bit flag preventing future unpinned accesses.

How would that even work? borrow and borrow_mut don’t know if they are being called in a pinned or unpinned way.

pretty much all interior mappings (including field access) are unsafe

Ideally they wouldn’t be. But we still have the same problem around Drop as Pin.

a pinned type could expose a &RefCell<Pinned<T>> and enforce the contract (“no &RefCell<T>”) through encapsulation

That would amount to the wrapping type upholding the invariants of RefCell. What I was looking for is a safe RefCell. It seems that to get that with Pinned, we have to either still duplicate the API surface, or we have to remove the Deref.

The conflation with “unsized” doesn’t actually make sense. Something without a dynamically known size can’t be deallocated, for example. This would imply that we leak all generators, and there’s no reason for it.

Ah, I forgot deallocation. Good point.

I was about to say that only some parties will forget the size, because we are casting pointer types around, but whoever is responsible for deallocation still knows the size. For example, someone would own a RefCell<T> (e.g. in a stack pinning API) but provide a &mut Pinned<RefCell<T>> to others. Deallocation is not affected.

However, PinBox is affected. We can’t actually define it as Box<Pinned<T>>. For every pinned object, there has to be something still holding it at the original type for the purpose of deallocation. So, PinBox would still have to be a separate type I think. But, in contrast to Pin, everything that works with &mut T where T: ?Sized would work with pinned data “for free”.

[withoutboats]

I’m not sure, but wouldn’t the allocator know the real size of the value?

No, our dealloc API passes the Layout so the allocator doesn’t have to know.

But, in contrast to Pin, everything that works with &mut T where T: ?Sized would work with pinned data “for free”.

This seems like a very small gain. I don’t know any methods like this I would want to pass a generator to in the first place - generators implement no traits, after all, and you can’t do almost anything at all with &mut T where T: ?Sized.

[RalfJung]

This seems like a very small gain. I don’t know any methods like this I would want to pass a generator to in the first place - generators implement no traits, after all, and you can’t do almost anything at all with &mut T where T: ?Sized.

Pinning has uses way beyond generators, as the experiments with intrusive collections show. We shouldn’t unnecessarily narrow this to just one use-case.

[withoutboats]

My opinion is unchanged thinking about other use cases. I don’t think you can build compelling generic use cases where you can pass a Pinned<T>. They’d have to be use cases where you have an &mut to something you know literally nothing about. I could imagine concrete use cases for e.g. &mut Pinned<SomeType>, but then you’ve gained nothing over PinMut<SomeType> because the pinning is still showing up in the API.

[comex]

Given the amount of discussion on this and other pin-related threads, I feel like I really need to come up with a proper proto-RFC based on the new idea, if I have any chance to make it a reality…

…But for now, to respond to @withoutboats:

I could imagine concrete use cases for e.g. &mut Pinned<SomeType>, but then you’ve gained nothing over PinMut<SomeType> because the pinning is still showing up in the API.

The generic use case would look like

fn takes_mut_ref<T: ?Sized>(t: &mut T)

or

trait Foo {
   fn takes_mut_self(&mut self);
}

T or Self (respectively) would be instantiated with Pinned<SomeType> – but of course, if you don’t need immovability, it could be any other type. Thus, pinning would not be “showing up in the API”.

The conflation with “unsized” doesn’t actually make sense. Something without a dynamically known size can’t be deallocated, for example. This would imply that we leak all generators, and there’s no reason for it.

For now, as @RalfJung said, PinBox would still be a separate type, and it would use the original type’s deallocation routine.

Nevertheless, the issue you mention is a good justification for having trait Move: DynSized. This way, if, say, a future language extension allowed immovable types to be created by value using ‘placement new’, code compatible with that extension could bound on DynSized but not Move.

Now, I think I was wrong to say that Move could be deferred added later as a backwards compatible extension. In particular – having a fundamental function like size_of_val sometimes panic is not a great situation, so even under just the DynSized RFC, we’d probably want to create a new function that does the same thing as size_of_val, but with an actual DynSized bound, and guaranteed not to panic. But if this new function came with a semantic guarantee (implicit or explicit) that you could use the returned size to move the value, then we’d end up having to change it again if we later made Pinned<T> impl DynSized (or added other types that impl’d DynSized but not Move).

So I think the initial design ought to include a Move trait. There wouldn’t need to be any special compiler semantics associated with it yet, i.e. forbidding moves of !Move types, because for now there would be no way to obtain !Move types by value to begin with. The only effects of Move would be:

• the existing size_of_val would panic for !Move types, and
• unsafe code using the new version of size_of_val would be expected not to use it to move objects around unless it separately verified T: Move.

[steven099]

How would that even work? borrow and borrow_mut don’t know if they are being called in a pinned or unpinned way.

Right, that’s the other half of it. Potentially you could use specialization, but using it to change behaviour like this could be considered abuse.

That would amount to the wrapping type upholding the invariants of RefCell. What I was looking for is a safe RefCell. It seems that to get that with Pinned, we have to either still duplicate the API surface, or we have to remove the Deref.

If Pinned<T>: Deref<Target=T> only where T: Unpin, then this would go away. Of course, you lose the ability to perform existing & operations on pinned values. Though, since field projection is already unsafe, maybe the right approach is to make &Pinned<T> → &T unsafe, in which case your coercion can be safe, giving you safe RefCell in a pinned context.

Restricting Deref for Pinned<T> could be a significant loss, since, as @withoutboats pointed out, Pinned<T> doesn’t implement any traits and can’t really be used for much on its own. Of course, types that are !Unpin don’t actually exist yet and likely would have operations defined on Pinned<T>—you can, after all, implement traits for Pinned<T> or use it as part of a custom self type. But it’s hard to think about the true impact of this in a vacuum of actual code using the feature.

It did occur to me that Deref might not be the right mechanism for this. What put the thought into my mind was the hypothetical &Pinned<RefCell<T>> → &RefCell<Pinned<T>> coercion you were exploring. But also, I feel as though the problem with field projection ties into all of this: the issues of not being able to do much with pinned values are somewhat underscored by the fact that you can’t even safely do field projection (except immutably via Deref at the moment). All of this led me to consider a not-proposal that I’ll refer to as 'deeply-magical Pinned’ (DMP).

Deeply-Magical `Pinned`

This is a not-proposal because it’s a very large idea which is outstrips even the ?Move proposal in its potential to be disproportionately complex. Also, it’s not really fleshed out; references/indirection aren’t considered, for one thing. However, I think it’s useful for thinking about the implications of design decisions. So far, we’ve considered Pinned<T> to be a flat wrapper that erases the movability of T. But we could instead consider it a type function that maps a type to its pinned equivalent (like your RefCell).

• Pinned<T> = T where T: Unpin

• Structures (leaving them anonymous for now):
  Pinned<{t: T}> = { t: Pinned<T> }

  This is consistent with Unpin and auto-trait poisoning, since:
  { t: Pinned<T> } = {t: T} iff T: Unpin

• Generic structure:
  struct S<T: ?Move> { t: T }
Pinned<S<T>> = { t: Pinned<T> } = S<Pinned<T>>
  (This also gives you Pinned<RefCell<T>> = RefCell<Pinned<T>>)
  (You can’t do this with a body like { x: T, y: InvariantlyUnpin<T> })

The interesting case, then, is when you have a structure with a (non-generic) !Unpin field:

struct S { x: SomeNotUnpin }

Since Rust doesn’t have structural identity for struct types, you can’t really express the pinned equivalent:

{ x: Pinned<SomeNotUnpin> }

except as Pinned<S>. This means that Pinned<S> can’t have any methods that aren’t explicitly defined on Pinned<S>; this is where restricted Deref for Pinned<T> would probably hurt the most. If you wanted to define methods that work regardless of whether S is pinned, you’d need something even more magical:

impl<const isPinned: bool> MaybePinned<S, isPinned> { ... }

This highlights the inherent limitation of ?Move: it works fine on type parameters to let code work with both pinned and unpinned types, but it won’t let you work with both the pinned and unpinned versions of a concrete type. I’m not sure how to draw implications from this into a concrete design, but it does narrow down the problem somewhat.

[withoutboats]

The generic use case would look like

fn takes_mut_ref<T: ?Sized>(t: &mut T)

or

trait Foo {
   fn takes_mut_self(&mut self);
}

Exactly, my claim is that there is very little you can do with this. In the first case, there are essentially no useful functions I can think of that can be implemented over a mutable reference to any type.

The second gets more to the point. What needs to implement Foo in this case is Pinned<MyType>, which means there needs to be an impl somewhere that cares about Pinned. This makes it not significantly different from the existing Pin API - someone could just as easily implement Foo for Pin<'a, MyType>.

I don’t see how this has bought us anything substantial.

[withoutboats]

Given that making predictions about the future is hard, the question this raises is: is “Approach A” forwards-compatible with “Approach B”, or can we make it be? That is, can we later add Pinned<T> and ?Move, declare that type PinMut<'a, T> = &'a mut Pinned<T>, and everything keeps working? If that is the case, then we can adopt “Approach A” for now with no reservations, and defer the question of ‘upgrading’ to “Approach B” into the future when we will know more about our requirements.

For what its worth, I believe that this is true. If it turns out that this proposal does open up more opportunities, I think we can make Pin a type alias for references to Pinned types.

[RalfJung]

If it turns out that this proposal does open up more opportunities, I think we can make Pin a type alias for references to Pinned types.

The one source of trouble here is the Deref impl, which is in conflict with Pinned<_> acting “uniformly” on owned and shared data as I spelled out above.

[steven099]

More broadly:

• The traits listed here could be a hazard. Without a blanket implementation for Pin<T>, downstream crates could create implementations on Pin<SomeType>, which could become conflicting if Pin<T> became &mut Pinned<T>.

• Deref and DerefMut for Pin would change their targets from T to Pinned<T> based on the blanket implementation for &mut, which applies even if both are restricted to T: Unpin.

• The inherent methods on Pin cannot cleanly be migrated to &mut Pinned, since &mut doesn’t have inherent methods. They could be put on an extension trait, though… Putting them on Pinned instead would be a breaking change, but fairly minimal.

• The methods on PinBox would conflict with those on Box. Keeping PinBox as a distinct type would probably hurt a lot less than keeping Pin, though, so this isn’t huge.

All of these are fairly minor as long as such a migration happens before pinning is stabilized. After stabilization, they could present a significant stumbling block if changing to Pinned was desired.

[RalfJung]

The methods on PinBox would conflict with those on Box. Keeping PinBox as a distinct type would probably hurt a lot less than keeping Pin, though, so this isn’t huge.

PinBox would have to stay separate anyway, as discussed above. Bon<Pinned<T>> cannot be deallocated.

[steven099]

That depends on whether you stick with Pinned<T>: !DynSized. If you split Move from DynSized, then I think Box<Pinned<T>> could be deallocated.

[comex]

In the first case, there are essentially no useful functions I can think of that can be implemented over a mutable reference to any type.

(I guess a more realistic example would have T: ?Sized + SomeOtherTrait where SomeOtherTrait has methods taking &mut self.)

What needs to implement Foo in this case is Pinned<MyType>, which means there needs to be an impl somewhere that cares about Pinned. This makes it not significantly different from the existing Pin API - someone could just as easily implement Foo for Pin<'a, MyType>.

The difference is that the trait doesn’t have to care about Pinned, which is important since traits can be implemented for many different types, including both immovable and movable ones. In particular, when you create a trait, you don’t have to decide between taking Pin<Self>, which works best for immovable types but has poor ergonomics for the foreseeable future and is inconsistent with the rest of the ecosystem, and taking &mut self, which may still be compatible with immovable types via &mut Pin<Foo> but with reduced performance and less functionality (doesn’t work for methods that return references).

edit: and I claim this is more forward compatible with a hypothetical future in which we have native immovable structs and can use them directly with, e.g., the standard Box type; no need for any types with Pin in the name.

[comex]

That depends on whether you stick with Pinned<T>: !DynSized. If you split Move from DynSized, then I think Box<Pinned<T>> could be deallocated.

Hmm, good point. I guess that we could just have a conversion from Box<T> to Box<Pinned<T>>, which, if you squint, looks a lot like an unsize coercion (Box<T> to Box<Trait>).

[RalfJung]

That depends on whether you stick with Pinned<T>: !DynSized. If you split Move from DynSized, then I think Box<Pinned<T>> could be deallocated.

I don’t think that’d work. This entirely proposal rests of DynSized: Move. After all, if Pinned<T>: DynSized, then (once we have unsized rvalues in return position) nothing stops you from moving that out of the Box.

[steven099]

The DynSized proposal hasn’t been accepted yet (afaik). size_of_val needs to be bounded by ?Sized + DynSized, but anywhere that wants to move dynamically sized values could instead by bounded by ?Sized + Move, with Move: DynSized. Move would have to be a default bound for all T: Sized.

(In my proposal up the page, this would have been ?Sized + DynMove, with Move: DynMove + Sized and DynMove: DynSized. But it you can get away with one less marker trait, why not? The only reservation I have is that having Move be a default bound for all T: Sized feels like less of a hack if Move: Sized, but having a salve for my conscience probably isn’t worth an extra marker trait.)

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.