Pin ergonomics

(NOT A CONTRIBUTION)

The attached gist contains a set of features which would work to make Pin more ergonomic.

The features specifically are:

  1. New pinned reference operators &pinned [place] and &pinned mut [place]
  2. Adding pinned references to method resolution.
  3. Assigning to pinned references.
  4. Pinned method receiver parameters (&pinned self and &pinned mut self).
  5. Supported pinned field projections with #[pinned_fields] attribute.
  6. [optional] DerefPinned operator trait.
  7. [optional] native syntax &pinned T and &pinned mut T types.

The only semi-breaking change would be the need to select a keyword for the new reference operators; I chose pinned in this gist because pin is used frequently in std APIs.

I may have made mistakes in my reasoning or understanding of e.g. method resolution; very eager to receive corrections.

My opinion is that adding some of these features would be a much easier way to solve the problem of pin ergonomics than trying to deprecate pin and add a new trait like Move. The biggest problem with Move is that it is not backward compatible to add either as an auto trait or as a ?Trait. I wrote about this in a blog post last year: Changing the rules of Rust

11 Likes

Isn't DerefPinned redundant? Isn't that always covered by the unconditional Deref impl on Pin?

2 Likes

(NOT A CONTRIBUTION)

Yea, that's probably true, because you can get there with the other change to method resolution by doing the normal deref on Pin<P> to get P::Target and then adding the &pinned operator.

EDIT: Updated the gist to reflect this. For those reading this conversation later, there were previously both DerefPinned and DerefPinnedMut in the gist; since the immutable one was unneeded, I rewrote the gist to add only the mutable version under the name DerefPinned.

Previously by me: Pin projection with lots of language support

4 Likes

(NOT A CONTRIBUTION)

I guess I forgot about your post; other than syntax there doesn't seem to be a lot of difference between the two.

(NOT A CONTRIBUTION)

Iterated on the ideas in this gist into a blog post here: Pinned places

14 Likes

How to construct a place that is both boxing and pinned, something like Box<#[pinned] T>, and how to construct this value (this still cannot avoid the issue of placement).

2 Likes

It avoids the issue of placement because T can always be moved until it is moved into a pinned place (and it doesn’t implement Unpin). T is not a place.

let pinned mut stream = Box::new(make_stream());

thank you for your recent posts, they've helped me comprehend pinning to the point I have questions :sweat_smile:

I might just not have read up enough and comprehended things, but I don't understand what a non-mut &pinned x reference can do?

(NOT A CONTRIBUTION)

You still use Box::pin (or Pin<Box<T>>::from) to construct Pin<Box<T>>. This part of the Pin API is unchanged.

Unfortunately this doesn't work because the Unpin impl on Box means that pinning a box doesn't pin its content. This impl was probably a mistake, but one that can't be rolled back now.

Very little. The only use case I know is the pin-cell crate, which I wrote a long time ago: crates.io: Rust Package Registry

You can't pin-project through a RefCell, because it gives you full mutable access to the interior. PinCell is an alternative cell which only gives you pinned mutable access to the interior. &pinned T is used in the API because it proves you can never move T again.

1 Like

Hm, doesn't seem like a very popular thing to do based on looking at the reverse dependencies on that crate.

But this means that in theory you need a PinCell and a PinMutex (and they should arguably be in the standard library). But since I haven't really heard complaints about the lack of those I assume they are just very uncommon operations.

(I haven't really had to do much with pin yet, but I found your two most recent posts very enlightening, thank you)

I suspect that, out of all the possible uses of pinning, almost all of the ones actually written so far are Futures (and related traits like Stream or AsyncRead that aren't strictly futures themselves), because it has been difficult to actually define a type that directly benefits from pinning (and is sound) other than a compiler-generated async block type.

And, in particular, putting a Future in a PinCell only makes sense if you want to share ownership and polling of the future, which is already provided by FutureExt::shared(). There's not a lot of opportunity for variety in exactly what you do with the Future, since it only has the one operation which has (from an application rather than scheduling perspective) zero inputs and one output.

Hopefully, UnsafePinned will help with that, by giving a well-defined foundation for “how do I soundly make use of my data being pinned”, and we’ll have a wider variety of uses of Pin in the future, and more reasons to use tools like pin-cell.

1 Like

Pin doesn’t have transitivity.:thinking:

So, we still need to use the Pin API, but introduce a new operator and build-in type &pinned {mut} makes the Pin API more user-friendly.

I guess:

let old_pinned_ref: Pin<&mut T> = blahblah;
let new_pinned_ref: &pinned mut T = &pinned mut * old_pinned_ref; // reborrowing by DerefPinnedMut.

Pinning is also widely used by crates that implement self referential structures. They use lots of unsafe, and it's indeed difficult for them to produce sound code (lots of crates in this space had their share of soundness issues), but it's theoretically possible to write sound self-referential structures in Rust using Pin.

it's hard to come up with a use for pinning that isn't related to self-referential data types in some way though (either compiler generated like async rust, or provided by a library using unsafe code)

edit: oh here's a cool use for pinning: the moveit crate implements C++-style move constructors, by forcing your objects to be pinned and then "moving" them logically using a move constructor (which is not a Rust move, since in Rust moves are always a memcpy; so to represent C++ move semantics in Rust you must use Pin)

Is it? As far as I know, none of ouroboros, yoke, or self_cell (the maintained self-reference crates I am aware of) use Pin in their API or implementation. They arrange for things to be immovable, but they don't expose this guarantee via Pin, only via borrow lifetimes. (And therefore, they all involve the referenced data being in some heap-allocated container internal to the self-referential struct, whereas Pin has the potential for zero internal allocation.)

I'd certainly like to see that happen, but as far as I've heard, it hasn't.

3 Likes

I think it is fine for futures alone, but Pin problem is not a futures problem. From my experience of writing pinned (not always self-referential) structs, is that you want them to pinned immediately after creation. With today's Pin you need to first create it, then pin, then initialize. The problem is, how to differentiate is a type initialized or not? You cannot change it's type after it's pinned. I'm transmuting references to a new type sometimes, but it has a cost - you cannot rely on drop. Maybe with some kind of &own this might work out. But again, for me main problem with Pin is not syntax and not unsafe code, but that you cannot create a good api with it.

Example:

use os_sys::RawSemaphore;
use os_sys::create_semaphore;

struct Semaphore {
    raw: UnsafeCell<MaybeUninit<RawSemaphore>>,
}

unsafe impl Sync for Semaphore {}
unsafe impl Send for Semaphore {}

impl Semaphore {
    const fun new() -> Semaphore { ... }
    fn initialize(self: &pin mut Self) {
          unsafe { create_semaphore(self.raw.as_mut_ptr())
    }
    fn get(&pin self) {
        // How to ensure that it is called after `initialize`? 
        // We cannot use typestate pattern and consume `self` in `initialize` to return new type. 
        // Maybe we can have signature like `init(&pin mut UninitSem) -> &pin mut Sem` but there a 2 problems:
    // 1. You cannot have Drop to delete semaphore - should I have a bool inside? Meh
    // 2. You would cry if would like to have semaphore inside another struct with this appoach
// !Move types solve that problem *perfectly*
    }
}
3 Likes

why do you write "(NOT A CONTRIBUTION)"?

1 Like

To expand on this, I seem to remember hearing that all communication to a project using Apache 2 as licence is automatically licensed under Apache 2 unless otherwise marked. (So yes, everything we write here is licensed under Apache 2 and could be reused by the Rust project.)

I may have misunderstood / misremember though.