Unsafe Impl Copy and Freeze

While it is certainly niche, the inability to unsafely impl Copy and Freeze seems to have recently actually been a barrier to some code I've been trying to write.

Now, obviously given the compiler integration with these traits, impling them is massively unsafe.

However, the inability to declare both an instance of Drop and an instance of Copy is actually a barrier for some (also massively unsafe) code.

In essence the place it has been coming up has been with cases where I basically need to manually implement drop glue due to some details. While the exact details are unimportant, here is a simplified example.

Suppose rust did not have native enums, and I was trying to implement an option type like so:

struct MyOption<T> {
    active: bool,
    data: MaybeUninit<T>,
}

impl<T: Clone> Clone for MyOption<T> {
    fn clone(&self) -> Self {
        if self.active {
            return MyOption {
                active: true,
                data: MaybeUninit::new(unsafe { self.data.assume_init_ref() }.clone()),
            };
        } else {
            return MyOption {
                active: false,
                data: MaybeUninit::uninit(),
            };
        }
    }
}

impl<T: Copy> Copy for MyOption<T> {}

impl<T> Drop for MyOption<T> {
    fn drop(&mut self) {
        if self.active {
            unsafe { drop_in_place(self.data.assume_init_mut()) }
        }
    }
}

I claim this is perfectly sensible code, though unnecessary in this particular case. Notably, this does not require a few guarantees that would be potentially problematic to give:

  • That drop is called when copy is satisfied
  • That drop is not called when copy is satisfied

Not requiring these guarantees means that the compiler does not have to be careful about what is a move vs a copy. Perhaps there will be extra calls to drop in the generated code. They are no-ops (unless T itself impls both Copy and Drop and has a drop that is itself not a no-op, which is precisely the thing that would be sketchy, and would I believe prove equally problematic to the enum Option).

This is the primary issue of importance to me with Copy, though I note the inability to impl Copy on a type without a drop implementation including drop glue if it wasn't impled on one of the inner types is a hole in what unsafe can easily do. If a library author doesn't want to promise that a type will continue to be copyable and so hasn't declared it, there is to my knowledge no remotely sane way to declare that your type which uses it is Copy. Such a thing would obviously be a hazard when updating dependencies, but doesn't seem as though it must be impossible.

As for Freeze, I do not believe there is currently any way to take a type variable T and have a field that has all the same niches and so on but also benefits from Freeze optimizations. Alternatively, a UnsafeAnticell<T> (bikeshedding probably required) that unconditionally implements freeze but shares the representation may be a better solution, as it can more safely encapsulate the unsafety of even giving out a &T. The meaning of such a type however seems quite straightforward. UnsafeAnticell<UnsafeCell<T>> = T. UnsafeAnticell<Option<UnsafeCell<T>>=... actually this one is a bit more complicated, but with a NoNicheWrapper it becomes UnsafeAntiCell<Option<NoNicheWrapper<T>>.

This is also relevant for some nonsense where I can guarantee immutability of certain regions of memory based on more complicated criteria. As Freeze is primarily an optimization, it is probably less of an issue than Copy, though I will note that const stuff depends on Freeze.

The Copy issue can also be resolved with a wrapper type naturally, though I think that is less natural than UnsafeAntiCell type things.

This may be technically the case in one sense of the word — it would not be incorrect to treat every type as if it were !Freeze for opsem purposes — it's extremely not true in the more generally useful sense, in that Freeze is fundamentally linked into the operational semantics of what it means to execute Rust code. Allowing manual impls to effect this is not something we want to allow until at least we are very sure exactly what that means for the opsem.

Breaking the implication that Copy means needs_drop() == false is also a very dangerous change, although for different reasons.

(I am a member of T-opsem, but this is my own (informed) position, not a position of the team.)

1 Like

I am specifically proposing that the implication that Copy means needs_drop() == false is not changed for this. That unsafely impling Copy does in fact risk your drop code being skipped.

But observe how with Option<T> it is allowed to impl Copy and impl something like Drop in the form of drop glue. These happen in a potentially mutually exclusive fashion, but dropping an Option<T> I believe may or may not read the discriminant even when the drop is itself a no-op (I believe this may happen around things like Option<dyn Foo>, but maybe actually it checks the vtable to see if it needs drop first? Either seems fine though, which is the key point.)

This cannot currently be in any way known to me duplicated in user code. Specifically the request is for a way to handle at least things where are 'drop-glue-like', where if the type is copy it doesn't matter whether or not it gets called.

Obviously generally compiler checking that things are 'drop-glue-like' would be a large demand, but that is what unsafe is for.

I think it’s fair and simple enough to say unsafe impl Copy means “if a type implements Copy, the compiler is permitted to not call Drop, as if inserting std::mem::forget; libraries are also permitted to assume Copy implies !needs_drop”. That said, I wonder how many of the motivating use cases would be satisfied with Autoclone/Claim/whatever being adopted. (And I’d like to see some of those use cases, the Optional one is a good theoretical one but it is also already taken care of by another language feature.)

This has come up for me in two primary areas. One is experimenting with baaasically writing some manual #reprs. Most of those have proved unsound in practice admittedly, especially the implementations for zero overhead sum of products representations for instance deriving. Another is with sort small-string like optimizations for some generic datatypes.

But I'm pretty sure both of these should be possible to do in a sound way with sufficient caution and avoidance of standard reference types.

Another case was actually trying to implement a more limited version of the UnsafeAntiCell<T> by storing things specifically as an array of MaybeUnit<u8>. I don't actually know if that one is sound, as it is based on questions about the implications of Freeze and mut during drop. Which I suppose is an example reason to say "wait until we are very sure exactly what that means for the opsem." for that one.

1 Like

The risk I see — which may or may not be what @CAD97 was thinking of — is not that drop glue might be skipped, but rather that libraries may have assumed that if they present a T: Copy bound to their users, then that implies “values of type T never execute any code when dropped”. Such constraints are sometimes necessary for the soundness of unsafe code, and so removing what was previously a way to enforce them potentially causes unsoundness.

Personally, I'd like to see an unsafe impl Copy that lets you skip the requirement that all fields must implement Copy, but not the requirement that no fields have drop glue. This would (in the most general case) be a post-monomorphization error since “doesn’t have drop glue” isn't expressible as a bound, but that's not too bizarre because unsafe code can already cause linker errors.

2 Likes

OK according to A type that is sometimes Drop and sometimes Copy: is this even theoretically possible? it is actually maybe already possible at least in some cases using the (also very unsafe) specialization system.

Note "values of type T never execute any code when dropped" if weakened to "have no side effects when dropped" would be fulfilled by the example MyOption, though I suppose even low but present side effect level things like panics could mess up unsafe code.

Note that these two statements are not equivalent. Actually making needs_drop() == false when the type implements Copy has the same issues of specialization, so probably cannot be guaranteed in general unless you somehow restrict when Copy can be implemented with Drop to be an always applicable impl.

What you can probably guarantee/require instead is that if a type is Copy then either it has no drop glue (needs_drop() == false) or its drop glue is effectively a noop (which would allow needs_drop() == true, but shouldn't break as many assumptions)

Ah. Fair enough. In the cases that I've been playing with it comes out to equivalent to drop glue though, so it might be possible to guarantee something stronger there?

Also I hacked together this code. Untested and VERY danger I expect, but it... is interesting:

#![feature(specialization)]
use std::marker::PhantomData;
use std::mem::transmute_copy;

#[repr(transparent)]
struct WithDrop<T: ?Sized, DropCode: DropCodeFor<T>> {
    _phantom: PhantomData<DropCode>,
    inner: T,
}

trait DropCodeFor<T: ?Sized> {
    fn drop_code(arg: &mut T);
}

impl<T: ?Sized, DropCode: DropCodeFor<T>> Drop for WithDrop<T, DropCode> {
    fn drop(&mut self) {
        <DropCode as DropCodeFor<T>>::drop_code(&mut self.inner)
    }
}

trait CopyDropHelper<DropCode: DropCodeFor<Self>> {
    type Inner: ?Sized;
}

impl<T: ?Sized, DropCode: DropCodeFor<T>> CopyDropHelper<DropCode> for T {
    default type Inner = WithDrop<T, DropCode>;
}
impl<T: ?Sized + Copy, DropCode: DropCodeFor<T>> CopyDropHelper<DropCode> for T {
    type Inner = T;
}

#[repr(transparent)]
struct CopyDropWrapper<T: ?Sized, DropCode: DropCodeFor<T>> {
    inner: <T as CopyDropHelper<DropCode>>::Inner,
}
impl<T: ?Sized, DropCode: DropCodeFor<T>> CopyDropWrapper<T, DropCode> {
    pub fn new(val: T) -> Self
    where
        T: Sized,
        <T as CopyDropHelper<DropCode>>::Inner: Sized,
    {
        unsafe { transmute_copy::<T, Self>(&val) }
    }
    pub fn into_inner(self) -> T
    where
        T: Sized,
        <T as CopyDropHelper<DropCode>>::Inner: Sized,
    {
        unsafe { transmute_copy::<Self, T>(&self) }
    }
    pub fn get_ref(&self) -> &T {
        unsafe { transmute_copy::<&Self, &T>(&self) }
    }
    pub fn get_mut(&mut self) -> &mut T {
        unsafe { transmute_copy::<&mut Self, &mut T>(&self) }
    }
}

impl<T: Clone, DropCode: DropCodeFor<T>> Clone for CopyDropWrapper<T, DropCode>
where
    <T as CopyDropHelper<DropCode>>::Inner: Sized,
{
    fn clone(&self) -> Self {
        CopyDropWrapper::new(self.get_ref().clone())
    }
}

impl<T: Copy, DropCode: DropCodeFor<T>> Copy for CopyDropWrapper<T, DropCode> {}

You'll get the issue if you have something like impl Copy for Foo<'static>, then the compiler will need to either report needs_drop() == false for Foo<'a> with 'a != 'static or needs_drop() == true for Foo<'static> even though it is Copy.

Wouldn't the former be fine, as the drop would have to only apply for non-static and hence must be safely skippable?

I could imagine an API where:

  • Foo<'a> is invariant in 'a
  • Foo<'static> implements Copy
  • there is a constructor for Foo<'static> that sets an internal actually_drop flag to false
  • there is a HRTB callback-based API to get a Foo<'non_static> that sets an internal actually_drop flag to true
  • Foo<'a> implements Drop and checks the actually_drop to determine whether it needs to drop anything

This is likely very unusual, however it does show a situation where:

  • Copy and Drop are both implemented
  • when Copy is implemented Drop is virtually a noop
  • it has specialization-like issues that prevent the compiler from determining whether a type implements Copy or not when needs_drop() is evaluated during monomorphization
  • it is incorrect to return needs_drop == false for Foo<'non_static>

Hence you need to make needs_drop() == true for Foo<'static> even though it implements Copy or change the rules under which an implementation of Copy overlapping with Drop is allowed, for example by requiring the overlap to be independent of lifetimes.