Smart pointer which owns its target

I published a new version 0.3.0 of deref_owned yesterday.

As said in my previous post, my main insight was that Box::into_inner is semantically different from IntoOwned::into_owned. All of

are already "owning" smart pointers. Note that there currently is no trait in std which classifies them as "smart pointers owning their pointee" or "smart pointers which could be converted into a smart pointers owning their pointee" (by a no-op/identity operation). They all implement Deref, but Deref is also implemented by &'a T.

Now &'a T isn't a smart pointer owning its pointee, but it's a reference which could be converted into a smart pointer owning the pointee if T implements ToOwned (and because a reference is Copy).

Defining IntoOwned

So let's see how we can find an abstraction for a pointer-like type that can be converted into an owned type while being consumed (which we will name IntoOwned).

First of all, we must demand that Deref is a supertrait, because Deref is what makes a type a pointer-like type. Moreover, such type must be Sized, because we want to pass it by-value to the conversion function. Hence we get:

pub trait IntoOwned: Sized + Deref

(as in line 91 in lib.rs of deref_owned)

Secondly, we need to know the type of the conversion result. Akin to ToOwned::Owned, this should be an associated type, such that we don't need to know the owning type (edit: what I mean is that we don't need to pick an owning type later when doing a conversion into an owned type; of course we still need to know it when implementing IntoOwned for a particular type). We want to be able to simply write .into_owned() without having to specify a type (same as when we use .to_owned()).

Here is how the associated type looks like:

pub trait IntoOwned: Sized + Deref {
    type Owned: Borrow<<Self as Deref>::Target>;
    /* … */

(as in line 93 in lib.rs of deref_owned)

By adding the bound IntoOwned::Owned: Borrow<<Self as Deref>::Target>, we demand that it's possible to get a borrow from the resulting type to our pointee (which is <Self as Deref>::Target), thus that Eq and Ord can be used consistently during all these operations. I.e. we say we must be able to get back again from the owned type to the borrowed type (this is just like the bound ToOwned::Owned: Borrow<Self>).

Finally, we need the actual conversion method:

    fn into_owned(self) -> Self::Owned;

(as in line 95 in lib.rs of deref_owned)

This consumes self and returns an owned type. The owned type (Self::Owned) isn't necessarily a pointer-like type (there is no Deref bound on IntoOwned::Owned), but we can go back from the owned type to the borrowed type using Borrow due to the previously added bound IntoOwned::Owned: Borrow<<Self as Deref>::Target>.

Implementing IntoOwned

Now let's see how IntoOwned is implemented in each case:

IntoOwned for &'a T

impl<'a, T> IntoOwned for &'a T
where
    T: ?Sized + ToOwned,
{
    type Owned = <T as ToOwned>::Owned;
    fn into_owned(self) -> Self::Owned {
        self.to_owned()
    }
}

(as in lines 98 to 106 in lib.rs of deref_owned)

We can only convert a reference to an owned type if the referenced type T implements ToOwned. hence we need a bound T: ToOwned. The conversion then happens using ToOwned::to_owned.

IntoOwned for Cow<'a, T>

impl<'a, T> IntoOwned for Cow<'a, T>
where
    T: ?Sized + ToOwned,
{
    type Owned = <T as ToOwned>::Owned;
    fn into_owned(self) -> <Self as IntoOwned>::Owned {
        Cow::into_owned(self)
    }
}

(as in lines 108 to 116 in lib.rs of deref_owned)

Here, we simply forward to the already existing method Cow::into_owned. Same as in the case of an ordinary reference, our result is <T as ToOwned>::Owned.

IntoOwned for Box<T>, Vec<T>, and String

These implementations are a no-op. That is because Box<T>, Vec<T>, and String are smart pointers which are already owned and where you can borrow their pointee, i.e. they already fulfill the bound Self: Borrow<<Self as Deref>::Target>.

Hence we simply do (example given for Vec<T> but it also applies to the other):

impl<T> IntoOwned for Vec<T> {
    type Owned = Self;
    fn into_owned(self) -> Self::Owned {
        self
    }
}

(as in lines 125 to 144 in lib.rs of deref_owned)

IntoOwned for T (such as i32) :interrobang:

But what if we have an owned value such as an integer? It is already owned. So there shouldn't be any problem, right?

i32 does not implement Deref<Target = i32>. It must not implement Deref because it is not a pointer-like type. But because it doesn't implement Deref, it cannot implement IntoOwned either :weary:.

This normally isn't a real problem because we just use i32 and we're happy. But when we do generic programming, we might not return a certain type T but a pointer-like type which points to T. Look at mmtkvdb::storable::Storable for an updated real-world example, which uses IntoOwned in a bound on a generic associated type (GAT):

pub unsafe trait Storable: Ord {
    /// Pointer to aligned version of `Self`
    ///
    /// This can be a simple reference ([`&'a Self`](prim@reference)) if there
    /// are no requirements for memory alignment, or can be a smart-pointer
    /// (like [`Owned<Self>`](Owned) or [`Vec`]) which drops the re-aligned
    /// copy when the smart-pointer is dropped.
    type AlignedRef<'a>: Deref<Target = Self> + IntoOwned;
    /* … */
    unsafe fn from_bytes_unchecked(bytes: &[u8]) -> Self::AlignedRef<'_>;
    /* … */
}

This trait is used, for example, by mmtkvdb::Txn::get:

fn get<'kr, K, V, C, KRef>(
    &self,
    db: &Db<K, V, C>,
    key: KRef
) -> Result<Option<V::AlignedRef<'_>>, Error> where
    K: ?Sized + Storable + 'kr,
    V: ?Sized + Storable,
    C: Constraint,
    KRef: StorableRef<'kr, K>, 

This looks horribly complex, but lets focus just on the (successful) return type Option<V::AlignedRef<'_>>. We basically return a pointer-like type to the stored value.

If we retrieve a string, then .get simply returns an ordinary reference to memory-mapped data on disk. But for some types like integers, we cannot do it becasue the memory-mapped data on disk will not have proper memory alignment (LMDB doesn't guarantee proper alignment). Here we must create a re-aligned copy of the data on disk. Freeing that memory should happen once the reference is no longer needed. This calls for a SMART POINTER!

And this smart pointer is exactly what Owned is. It is a wrapper which serves as a smart pointer to the wrapped value:

/// Smart pointer to owned inner value
#[derive(Clone, Default, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct Owned<T>(pub T);

impl<T> Deref for Owned<T> {
    type Target = T;
    fn deref(&self) -> &Self::Target {
        &self.0
    }
}

/* … */

(as in lines 22 to 31 in lib.rs of deref_owned)

In case of mmtkvdb, this is used to implement Storable for integers, such as i32:

macro_rules! impl_storable_fixed_size_force_align {
    ($type:ty, $optimize_int:expr) => {
        unsafe impl Storable for $type {
            /* … */
            type AlignedRef<'a> = Owned<Self>;
            /* … */
            unsafe fn from_bytes_unchecked(bytes: &[u8]) -> Self::AlignedRef<'_> {
                Owned(read_unaligned(bytes.as_ptr() as *const Self))
            }
        }
        /* … */
    };
}

(link to source)

Now let's get back to our IntoOwned trait. How do we implement it for Owned? It's trivial!

impl<T> IntoOwned for Owned<T> {
    type Owned = T;
    fn into_owned(self) -> Self::Owned {
        self.0
    }
}

(as in lines 118 to 123 in lib.rs of deref_owned)

To convert an Owned<T> into its pointee, we can simply extract the wrapped value of type T. Note that T fulfills the Borrow bound on IntoOwned::Owned because there is an implementation of Borrow<T> for any T.

Now how can we use this? Let's look at mmtkvdb again:

pub trait Txn {
    /* … */
    /// Get owned value from database
    fn get_owned<'a, 'kr, K, V, C, KRef>(
        &'a self,
        db: &Db<K, V, C>,
        key: KRef,
    ) -> Result<Option<<<V as Storable>::AlignedRef<'a> as IntoOwned>::Owned>, io::Error>
    where
        K: ?Sized + Storable + 'kr,
        V: ?Sized + Storable,
        C: Constraint,
        KRef: StorableRef<'kr, K>,
    {
        Ok(self.get(db, key)?.map(|x| x.into_owned()))
    }
    /* … */
}

(link to source)

Here .into_owned is used to convert the pointer-like type into the pointee. In case of an Owned<i32>, for example, this happens without unnecessary cloning (the cloning for memory re-alignment did already happen, we don't need to clone again).


I hope this made my motivations a bit more clear.

1 Like