Strong updates status

language design
42

Indeed, for types Foo<T> that have an indirection to the type T we want to update (otherwise no need for strong update), that are cheap to deconstruct and reconstruct (size_of::<Foo<T>>() is small), and store the type T in a way that only cares about layout, we can use strong update between Foo<MaybeUninit<T>> and Foo<T> to the cost of deconstructing and reconstructing (which may be optimized out).

However, I'm not convinced by the benefit. This permits to have init be a safe function. But to call it you need to provide the initializing closure, which needs to use unsafe because it will call assume_init_mut(). So it's not clear to me why prefer this compared to what we would do now:

/// Safety: f must initialize its argument
unsafe fn init(xs: Vec<MaybeUninit<T>>, f: impl FnMut(&mut MaybeUninit<T>)) -> Vec<T>
43

With strong updates the user may use an init function of the type (uninit here means the contents of the box are uninitialized):

let uninitial = Box::uninit();
let initalized = uninital.init(Foo::init);

But still such a general Foo::init api is rare (Huge structs/arrays, can't find any other use). Even for self-referential structs, constructed in-place behind a Pin, an unsafe api is ok, as self references currently require raw pointers. I think strong updates only truly shine together with other language features (at least partial types, where the prior could fully be safe ([T{} -> T]), but also some implementation of safe self references).

Alternatively rust could treat MaybeUninit specially and guarantee same layout of C<MaybeUninit<T>> and C<T>. That way an in place api is possible:

fn init(xs: &@a mut Vec<MaybeUninit<T>>, f: impl FnMut(&@b mut MaybeUninit<T>) -> &@b mut T) -> &@a mut Vec<T>

Similarly with repr(transparent) we could define the order, take any #[repr(transparent)] struct T, that contains S, as it is stored as an S, it has to be a valid S, [S..T] is possible. As a union MaybeUninit still is a special case, here the compiler needs to know that any T is a valid MaybeUninit<T> ([MaybeUninit<T>..T]). We cannot know that for any union, as it may arbitrarily constrain any variant. (Also it actually contains a ManuallyDrop<T>)

44

This is probably impossbile to implement. Option<NonZeroU32> relies on NonZeroU32 never having an all-zeroes bit pattern to work correctly. If Option<MaybeUninit<NonZeroU32>> were forced to have the same representation, then it would likewise rely on MaybeUninit<NonZeroU32> never having an all-zeroes bit pattern. But there are numerous safe APIs that would make it possible to put an all-zeroes bit pattern into a MaybeUninit<NonZeroU32> (because MaybeUninit is intended to be able to hold arbitrary bit patterns and the APIs are designed around that).

What you'd want instead is something that works like MaybeUninit but copies validity invariants from the wrapped type, i.e. it is capable of holding any sequence of bits that could be valid for a value of that type. This would probably require some sort of DefaultUninit trait that defined an arbtrary bit pattern for the type that was at least compatible with its validity invariants, even if it wasn't a valid alue of the type (e.g. the all-0s bit pattern is not a valid bit pattern for &'static u64, but 0x8 has a valid sort of bit pattern for the type even though it's unlikely to actually be a valid &'static u64 value).

45

Oh, forgot about niches, that makes things a lot more complicated. This applies to all cases: [Option<u8>..Option<NonZeroU8>] is also impossible because of it. So it would have to be way more specific, just have the types behind pointers (raw, NonNull, ref, etc.) not affect layout. That would make all Boxes, Vecs, etc. the same layout and enable remote init.

Thanks for the note, this is something I should consider for partial types (T{} needs to check for niches).

46

I just read the Immobile types and guaranteed destructors project goal and I think it would well with strong updates. The Forget and Destruct traits could enable strong update types (depending on the interaction between !Forget, !Destruct and unwinding), and the strong update types would help to deal with constructing types that don't implement Move. I also think it would help to add to std::mem:

fn strong_replace<'a, T, U, V>(dest: &'a mut [T..U], src: V) -> (&'a mut [V..U], T);

fn transform<T, U>(t: T, f: impl FnOnce(&mut [T..U])) -> U;

/// Identical layout to `T` but memory is not initialized
type Uninit<T>;

impl<T> Box<T> {
  deref_strong<'a, U>(self: &mut [Box<T>..Box<U>]) -> &mut [T..U];
}

This would also allow a safe replace_with implementation:

fn replace_with<T>(m: &mut T, f: impl FnOnce(T) -> T) {
   let (old, m) = mem::strong_replace(m, Uninit::<T>::new());
   let new = f(old);
   mem::strong_replace(m, new);
}
1 Like
47

Interesting, indeed that would be a first step (necessary to describe the properties of &'a mut [T..U]). The next would be support for borrowing, because borrowing a place of type T could now result in a place of type U once the borrow end. And this still has all the issues regarding T and U having the same validity invariant (including niches and layout within other types).

48

I think that while helpful, the ability to strongly borrow local variables wouldn't be necessary to make strong references worth while. Thinking about it more I think my old strong_replace could be split into:

fn strong_update<'a, T, U, V, R>(m: &'a mut [T..U], f: impl FnOnce(&mut [T..V]) -> R) -> (&'a mut [V..U], R);

fn replace<'a, T, U>(dst: &'a mut [T..U], src: U) -> T;

fn strong_replace<'a, T, U, V>(dest: &'a mut [T..U], src: V) -> (&'a mut [V..U], T) {
  strong_update(dest, |m| replace(m, src))
}

I also forgot to mention that &mut [T..U] should implement DerefMut<Target=T>

The we would at least be able to do something like:

let mut x = Uninit::<NotMoveType>::new();
let (m, _) = strong_update(&mut x, NotMoveType::new_in_place);
do_somthing(m.deref_mut());
strong_update(m, NotMoveType::destruct_in_place)

I'm almost thinking that strong mutable references could be (or at least start as) a library type with conversions to an from mutable references, the only difficulty would be telling the compiler that the type is only valid when both type parameters have the same layout.

I don't know how much of a concern niches an layouts within other types would be, I could see &mut [NonZeroU32, u32] being allowed but &mut [Option<NonZeroU32>, Option<u32>] not being allowed, I think you would need to prevent projections of strong references except for #[repr(transparent)] and #[repr(C)] structs. It could even start with no projections at all with derive macros instead.

49

I was wondering how you create strong references, but from your example it seems you can only create normal mutable references through borrowing, and then "enrich" them to strong references later (essentially viewing them as &mut [T .. T] and then possibly using strong_update() to change each type independently).

Yes exactly, I was considering that too, but missed the fact that it's not necessary to create them from borrowing a place, it suffices to create them from mutable references. I believe this could be possible, but I'm wondering how often this limited version is useful.

Indeed.

Indeed, if it's not possible to borrow a strong reference (only a mutable reference), then maybe niches won't be an issue, because you have to put back a valid value (thus avoiding niches) before the end of the original mutable reference.

This weaker version of strong updates (essentially just strong references without strong borrows) seems like a feasible first step as a library type.

1 Like