Comparison of wirhoutboats' and Niko Matsakis' recent blog series on improving pinning?

language design
1

Both @nikomatsakis and @withoutboats have recently made their own series of blog posts about improving the ergonomics of pinning in Rust, in radically different ways.

I feel there is a lack of comparison between these two proposals. What are the pros and cons of each? Are they competitors or complementary?

I asked on reddit and boats did answer what their view on it was: desiringmachines comments on Generators with UnpinCell

I would like to hear from the "other side" so to speak as well. Hopefully this is the right place to ask about that, I don't know what the right location to ask in would be otherwise. Currently it feels like there are two separate discussions going on.

References:

2 Likes
2

Say what you want about without boats but his proposal seems at least backwards compatible.

The other proposition seems like it will cause a Rust 2.0 type schism. Which I can understand but seems like small reason to cause major breakage.

5 Likes
3

As an update to this, it seems that Matsakis most recent post does address this comparison to some extent. At least it no longer feel like people talking past each other.

Link: MinPin: yet another pin proposal · baby steps

4

All that talk about pin ergonomics is great, but I am personally ok with having unsafe code under safe signatures.

What I don't like is being forced to have unsafe or unergonomic APIs. And lack of placed init (super let/view{} idk) and linear (or at least unleackable) types are really hurting. Embedded development :person_shrugging:

That whole Pin discussion seems like waste of discussion capacity to me ...

2 Likes
5

Problem with those things is that they are completely orthogonal to the issue at hand. async and Pin, and async generators and such have little to do with Leak trait. Also Rust needs to take into consideration other aspects, not just embedded development.

6

Yes. But the flipside is that sometimes it feels like people don't give enough consideration to embedded, or more generally all the other use cases for self-referential types besides futures. For example, Niko's MinPin blog post says:

  • Pin is its own world. Pin is only relevant in specific use cases, like futures or in-place linked lists.

and later:

So this is what I meant by “pin is its own world”: pin is not very interopable with Rust, but this is not as bad as it sounds, because you don’t often need that kind of interoperability.

This makes sense for futures. But the thing about an in-place linked list is that almost any object may want to be a member of an in-place linked list. Or may want to contain another object which itself is a member of an in-place linked list, and so on recursively. And those objects may want to implement almost any trait.

Making Pin its own world means that code that wants to use these data structures must itself live in its own world.

Maybe that is a necessary

function of where we are in Rust’s evolution

…but it's unfortunate.

That said, there aren't that many important generally-applicable traits with methods taking &mut self, so I may be taking the phrase "own world" out of context a little. But it's still a real issue.

8 Likes
7

(NOT A CONTRIBUTION)

Comparing overwrite with improving the UX of Pin is really difficult because they're just in different categories of language changes, but "minpin" is directly comparable to "pinned places." There is really only one key difference (every other difference is syntactic and not important) which concerns pin projections.

In the updated form of pinned places, pinned projections are permitted if a type meets these requirements:

  1. If it implements Unpin, it does so using the auto-trait mechanism, not a manually written impl.
  2. If it implements Drop, either it implements Unpin or its destructor uses the fn drop(&pin mut self) signature.

Niko replaces the first requirement that either the type being projected to implements Unpin or the type being projected through has an explicit negative implementation of Unpin. The aim here is to be more explicit, which is an admirable goal, but it means that strictly fewer pin projections are made safe, and the pin projections that are not made safe are critically important.

Specifically, Niko's rules do not adequately cover combinators, because combinators are Unpin iff the futures/streams they abstract over are Unpin. So in the implementation of Future for Join, for example, you do not know either that the joined futures are Unpin (because they may not be) or that Join is !Unpin (because it is Unpin if the joined futures are), and cannot use pin projections with Niko's rules.

Niko "solves" this by adding an explicit negative impl of Unpin for Join, but this is unnecessary for correctness and reduces the ways the combinator can be use, for no reason except to satisfy the requirements that Niko has imposed to give the implementer convenience features. Specifically, a Join of two Unpin futures should be moveable after it has been pinned, because it contains no self-referential state.

For future combinators this doesn't matter very much because you're unlikely to move a future after you start polling it, but for stream combinators it actually does: if you pin a stream to call next on it, and it implements Unpin, you can then move it later. There are plenty of plausible use cases for this, like special handling the head message from a stream and then looping over the remaining messages.

I doubt libraries would willingly make their API surface less useful than it has to be, so I would expect library authors to continue to maintain unsafe pin projection code instead of blanket negative implementing Unpin for their combinators (also this would be a breaking change). Since combinators are the primary use case for pin projections, this would make the feature far less useful.

The goal of making opting into pin projections explicit seems reasonable, but a different mechanism that does not involve unnecessarily restricting functionality should be found if that requirement is to be met.

9 Likes
8

Pin seems unnecessarily complicated.

I haven't read the literature about alternatives extensively but I am sure the notion of Scoped Pointer would have surfaced as a specialization of (Global) Pointer. Scoped Pointer could be implemented as a simple usize index into a host object tracted by the compiler as the optimized version. A less optimal solution is a tuple of a Global Pointer and an Index value. Conversion functions would be convenient.

9

Well, they are not orthogonal at all, as unleakable types will allow ergonomic structured concurrency, as well as many other patterns. Placed init is also tightly connected to futures and Pin in general - you just can't create a type in a pinned state / have typestate pattern.

2 Likes
10

Keep in mind, I'm talking about exclusively the quoted parts. Pin improvement, Unleak or in place initializing are not inter-related. As in: adding (Un)Leak wouldn't solve Pin ergonomics, nor would fixing Pin ergonomics solve Leak issue.

The problem is, adding unleakable types is a huge compatibility break. Maybe keyword generics can add it as an effect. But I would not hold my breath.

Note: this may or may not apply to structured concurrency, just unleakable types.

By tying something achievable in the short term (Pin improvements) to something very improbable in the long term (Leak types) people will get the worst of both worlds. Small improvements to be done never.

1 Like
11

You call this a pointer, but it isn't compatible with a "normal pointer" (e.g. dereferencing it is a completly different operation), so it cannot be used where a pointer is used today. Most notably it can't be used to model pointers inside async functions and blocks, so it isn't a replacement for Pin.

3 Likes
13

(NOT A CONTRIBUTION)

This discussion is off-topic for this thread, but it should be clear that the specific issue is that lifetimes cannot influence the representation of references because they are in a subtyping relationship, which means you need to be able to replace a reference with a longer lived reference without changing code. A reference that lives across await can be both a subtype and a supertype of references that do not, and need to be able to be substituted according to that relation, which requires that all of them have the same representation.

You "could" do this by having every reference ever have state to determine if it is an offset or a pointer, but imposing this branch on every dereference would be untenable for Rust's goals.

Monomorphization does not work here because you do not know at compile time whether or not a reference points into coroutine state because of this subtyping relationship.

4 Likes