Even more minimal placement new

language design
1

I recently had occasion to need placement new which sent me down the rabbit hole of reading the older and newer RFCs.

If the goal is to start with the most minimal RFC, I think it's possible to go even more minimal than the current RFC (leaving GCE for later) and still have a useful feature, and looking at the old RFC and the new one it seems like people may have been too eager to reject things on the basis of not solving everything at once. PoignardAzur is basically being asked by everybody to simultaneously solve the problem that rust doesn't have variadics/perfect-forwarding and add copy and move ellision rules. While high standards are good I think the desire to solve everything at once is blocking consensus.

C++ had placement new and users getting lots of value out of it for over 10 years before variadic templates and perfect forwarding were added and made it possible to generically/safely wrap construction. Placement new was in C++98, and originally there was no way to implement operations like emplace on vectors, and despite that it was still hugely useful for writing highly efficient code.

If Rust added C++ placement new as is, literally something like:

let foo: *mut Foo = unsafe { Foo (buffer) { field: make_value()? }; }

Equivalent to:

auto p = new (buffer) Foo(make_value());

What would this prevent adding in the future? Syntax can be bike shedded of course. The problem of wrapping generically and being able to support things like emplace for Vec items could be handled in later RFCs.

With this sort of approach all the concerns caused by closures and fallibility go away. break, continue, and the ? work exactly as one would expect. Likewise you are not left hoping the compiler inlines the closure calls you want it too (see my comment on the current RFC). I can't speak for serde/mio authors, and I have not tried writing a zero copy serialization framework in Rust, but I have written one in C++ before and this was as much as I needed. When you're iterating fields at compile time generating separate code for each struct you're serializing anyway it's not a big deal to generate a function that does the specific placement new you need and the problem of variadics and perfect forwarding can mostly be ignored.

With this approach the fact that you are doing a placement new is explicit by virtue of having a slight change in syntax in order to specify the buffer, and users don't need to memorize any rules to figure out whether or not placement new will actually happen. As far as I can tell having this as a built-in language feature wouldn't prevent adding GCE, or variadics, or safe wrapping APIs. I have not specifically investigated but I assume it also pretty trivially maps to LLVM. I'll resist the urge to speculate about what the implementation effort in rustc would be but I have to assume that it's less than any other proposal because it doesn't require the compiler to do anything smart.

Maybe if buffer is a &mut [u8] we can have placement new give back &mut Foo instead of *mut Foo where the lifetime is inherited from buffer. But the operation is going to have to be unsafe anyway because we don't know if we are clobbering some other object, and in the kind of context where you are likely to use placement new you are probably going to immediately turn a reference into a pointer anyway.

If we really wanted to bikeshed syntax, for the first version could just expose a variadic macro placement_new!(buffer, Foo, arg1, arg2); that has to be imported and that is directly implemented by the compiler. Then even downstream tools like rustfmt, syn/parse, etc. don't have to lift a finger.

2

Rust already has this. Here's a function that does the equivalent:

unsafe fn init_foo(this: &mut MaybeUninit<Foo>) -> &mut Foo {
    let ptr = uninit.as_mut_ptr();
    addr_of_mut!((*ptr).field).write(make_value()?);
    this.assume_init_mut()
}

It works, but it's very annoying to use.

The reason it's so annoying to use it, is that it requires the constructor and the function calling it to both be changed.

In C++, constructors are functions that take a this pointer and initialize the object behind it. Because of this, you can use placement new with classes that aren't explicitly designed to support it. Only one side, the side doing the allocation, even has to know that placement new is a thing.

In Rust, the idiomatic constructor pattern is just a fn new() -> Self. If this isn't going to work with placement new, then it means both sides need change what they're doing: not just the function calling the constructor, but also the constructor itself. In order to have a placement new feature that is at least as useful as the one in C++, where the function calling the constructor can use placement new without the object being allocated needing to do anything special, fn new() -> Self has to work with it.

If you don't care about supporting fn new() -> Self, then you already have all the ingredients that you need to roll placement new yourself. Start with a MaybeUninit<Self>, scribble over it with ptr::write, and call assume_init. It's completely unsafe, but it works.

9 Likes
3

I think there might still be large ergonomics gaps between the MaybeUninit version and what I propose. Because the rust ABI makes no guarantees, so type representation in memory may change depending on what it is nested inside of, I assume you would need to use addr_of_mut at the top level repeatedly drilling all the way down to the leaves like addr_of_mut!((*ptr).field.field.field);? Is there an easy way to express ..Default::default()? You also lose the field: value syntax.

Actually, with niches is it possible to sanely use the MaybeUninit strategy at all without everything being repr(C)?

4

You are allowed to write the full size_of<T> memory for the value. Niches only matter when there's not actually a T value, like an Option<T> that is None.

1 Like
5

Because the function signature of Default::default is fn default() -> Self, it either requires RVO, or it requires the return value of default to be copied into the placement new slot.

1 Like
6

I read the whole RFC comment, and while it looks like it’s currently blocked, I think that there is a path forward:

  • Make guaranteed RVO a thing. Implement the required modifications in the compiler.
  • Do not include any change to std. Maybe add a Box::with and Vec::push_with function being a perma-instable flag to gain knowledge.
  • Work on NRVO, and implement the required modification in the compiler as a second step.
  • Finally analyze if Box::with and Vec::push_with are really needed, or if a generator is better suited, …

In the meantime, it would make it possible to still get all the benefits of guaranteed RVO.

4 Likes
7

I suggest checking out Miguel's work on the moveit crate (and his recent conference talk).

This crate already implements placement new as well as supporting more generally self-referencial types by introducing constructors to Rust. The only thing left to do is to sprinkle it with a bit of sugar to improve ergonomics.

With his crate:

let foo = Box::emplace(moveit::ctor::new(Foo::new());
4 Likes
8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.