The broption crate and the status of placement new

[Soni]

The broption crate aims to propose an alternative design/approach to placement new. But we don't know where the rest of the language and users are at when it comes to placement new.

We created broption for use with selfref. But it's not quite the design we wanted. If we were to do it again, we'd make a design based on a similar principle to qcell::TCell, and enforce MaybeUninit instead of Option, but that has its own drawbacks - namely only being able to do fieldwise-initialization of a struct. We wouldn't be able to e.g. initialize a whole vec of structs. We don't think that's solvable, tho, so we'd rather stick to solvable problems.

Do you think this is an worthwhile approach? Do you think the language could be made to make this easier? Any other thoughts?

[afetisov]

It's not zero-cost (runtime checks + memory overhead + layout changes), so I don't think it fulfills the purpose that people want from language-level emplacement. It may be a nice temporary crate-level workaround, though with a different layout and no way to access the place of T I think that it's still very limited.

[Soni]

Yes indeed, the idea with the TCell-inspired variant is that it'd use MaybeUninit.

Honestly, we don't see the issue with runtime checks, especially when they get optimized out. For example, with broption, we hope the initialization gets optimized out, and then no runtime checks actually happen, tho it still brings about the memory overhead and layout changes. But what if we could get rid of the memory overhead and layout changes? (Well, as much as MaybeUninit lets us - sadly MaybeUninit does lose niche optimization...)

See, if we mark the branded maybe uninits with a type, like so:

struct BMaybeUninit<'brand, Marker: 'static, T: ?Sized> {
  _p: PhantomData<...>,
  inner: MaybeUninit<T>,
}

Then we can store the initialization state of each Marker in a separate struct - specifically, in the Factory. But we can only allow one Marker per Factory.

This loses the memory overhead and some of the layout changes. And the runtime checking still gets optimized out if LLVM is doing its job.

[Soni]

Ah, right, that wouldn't quite work.

Because Drop.

If having 'brand = 'static means known to be initialized, but you need to be able to construct uninitialized instances, then Drop cannot work properly. However, it could just be a lang item. Lifetime specialization (of Drop) is technically unsound, but let's do it anyway. Because leaking is safe.

What this means is that we're proposing placement new as a lang item. As far as we know, this is a completely different design from every other placement new proposal.

[afetisov]

I don't see any specific proposal here. "As a lang item" doesn't mean anything on its own, apart from "the compiler treats it specially".

While emplacement will certainly require compiler changes, they should be principled and minimal. I have trouble understanding what is that you propose, really, much less why this is supposed to be a flexible future-proof solution.

[Soni]

So the basic idea is to have as a lang item:

pub struct BrandedUninit<'brand, T: ?Sized> {
    inner: MaybeUninit<T>,
}

impl<'brand, T> BrandedUninit<'brand, T> {
    /// # Safety
    /// Must not expose an 'static brand unless it's fully initialized.
    pub unsafe fn new_uninit() -> Self;
}

// NOT 'brand!
impl<T: ?Sized> Drop for BrandedUninit<'static, T> {
    ...
}

This is because lifetimes are the only thing with the correct semantics, i.e. a for<'a> is opaque to 'a, while a 'static is transparent to 'static, and transmuting lifetimes is guaranteed (as per mem::transmute documentation) not to affect layout.

Then, it is possible to have arbitrary code that contains BrandedUninit<'a, T> similar to how BOption is handled. E.g.

struct Foo<'a> {
  x: Wrapper<'a, Something>,
}

let foo: Foo<'static> = ...;
drop(foo); // drops Something
let bar: Foo<'_> = ...;
drop(bar); // leaks Something

If you look at broption, it relies on dropping an Option, but what if it statically decided whether to drop something? What if initialization could be statically known?

Like uh, this would be UB, since the types are different:

trait InitState {
  type Inner<T>;
}

struct Uninit;
struct Init;

impl InitState for Uninit {
  type Inner<T> = MaybeUninit<T>;
}

impl InitState for Init {
  type Inner<T> = T;
}

Which means we can't expose an API like:

pub fn factory<T, F>(f: F) -> T::Kind<'static, Init> where
    T: Wrapper,
    F: for<'id> FnOnce(&mut Factory<'id>) -> T::Kind<'id, Uninit>,

So, at least currently, we must use an Option if we want Drop to ever be called. We want an alternative where we can still get to call Drop (under some circumstances) but without the overhead of Option.

An alternative to the impl Drop for Foo<'static> would be a pair of types (like the above trait InitState shows) which are guaranteed to monomorphize to the same layout, despite being different types. Basically something that makes it safe to transmute between them. This would also allow getting the drop behaviour without the Option overhead.

sorry for rambling we're just not sure how to convey the issues we ran into to others.

[Soni]

hmm maybe we want something to do with sealed traits? or a modified version of sealed traits?

importantly it'd need to be unsafe (or straight up disallowed) to specialize on these types - and that includes arbitrary trait impls.