Smart pointer which owns its target

language design
1

I wonder if it might be good to extend std to provide a smart pointer which owns its target. See deref_owned for an example implementation (originally implemented in mmtkvdb::owning_pointer).

/// Smart pointer to owned inner value
#[derive(Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Debug)]
pub struct Owned<T>(pub T);

impl<T> Deref for Owned<T> {
    type Target = T;
    /// Returns a reference to the contained value
    fn deref(&self) -> &Self::Target {
        &self.0
    }
}

impl<T> DerefMut for Owned<T> {
    /// Returns a mutable reference to the contained value
    fn deref_mut(&mut self) -> &mut Self::Target {
        &mut self.0
    }
}

This is basically a Cow which is always owned and never borrowed.

Some possible use cases:

2

First, generally we only put things in std after they're very successful on crates.io.

Second, you have not actually mentioned what problem this is supposed to solve. To me, wrapping things in Owned that just provides a Deref seems pretty pointless.

8 Likes
3

I didn't want to propose adding things to std right now. The crate has been published a few minutes ago, I'm fully aware of that. This is merely to start a discussion, not to make a pull-request or even an RFC.

Why std? Because we also have std::borrow::Cow. We also have a variant of it which is always borrowed, namely &'a T. But a variant which is always owned is missing.

Allowing to pass owned values where (generic) references are expected without resorting to Cow, which has runtime overhead.

The trait deref_owned::IntoOwned provides an abstraction over all three variants (Owned<U>, Cow<'a, T>, and &'a T) (edit: where T: ToOwned<Owned = U>), and performs cloning only where necessary (determined at compile-time).

1 Like
4

I'm looking for an actual use case. I'm also skeptical that the runtime overhead that Cow imposes causes significant problems in practice.

3 Likes
5

See links in my OP.

There are other things in std which allow us to reduce runtime overhead. To give a few examples:

1 Like
6

Just to clarify: Even if we don't want to add a new feature right now, talking about things which are potentially missing can help when adjusting other things in std or adding new method names or traits.

Note, for example, that deref_owned::IntoOwned::into_owned has already the same name as std::borrow::Cow::into_owned. Both provide the same functionality. Hence it would be difficult (or at least a bit ugly) to add a trait with practically the same method.

(edit: changed into_inner into into_owned)

Another example where abstraction was/is (potentially?) missing is TcpListener::accept and UnixListener::accept. The same method name, but different methods (i.e. there is no trait to provide an abstraction).

IntoOwned::into_owned would provide an abstraction to convert a pointer into its target (by consuming it, opposed to std::borrow::ToOwned::to_owned, which takes a shared reference).

7

What's the benefit of using Owned<T> instead of just T?

Is it just about the Deref impl, which all types don't have (even if they are "trivially" *&T -> T)?

12 Likes
8

How is Owned a smart pointer? When you say "owned smart pointer" I think of Box. I fail to see how Owned has anything to do with pointers. It's just a newtype for T...

16 Likes
9

It can be used where you want to be generic over P: Deref<Target = S> + IntoOwned<Owned = T>. Then P can be &'a S, Cow<'a, S>, or Owned<T>. I.e. an implementation can decide whether it returns a reference, a Cow, or an owned value. The caller which receives such P can treat all three cases generically.

For a concrete use case, see here (requires GATs, edit: in this particular use case, the GAT is defined here).

Yes, and about IntoOwned.

What do you mean? Can you explain?

It's a matter of definition. You may look at "What are smart pointers?" on URLO or Issue #91004.

In short: I consider it a smart pointer because it can be dereferenced. (But I understand if not everyone would share that opinion / use of wording.)

10

Maybe I was thinking too complicated there. If the Owned wrapper is really useless, I'm interested in how to do it without (and sorry for the noise here in that case). I believed it was the only way to provide such abstraction.

11

Rather than P: Deref<Target=T> + IntoOwned<T>, you want P: Borrow<T> + IntoOwned<T>. Borrow has a blanket impl impl<T: ?Sized> Borrow<T> for T.

3 Likes
12

Hmmm… :thinking: But Borrow doesn't support deref coercion. Also, how can I implement IntoOwned for T without creating a collision with implementations for &'a T and Cow<'a, T>?

Edit: Ah, I just noticed you made T a type argument to IntoOwned instead of an associated type. That would fix the collision maybe, but not sure if it creates other problems in my use case. Note that std::borrow::ToOwned::Owned is an associated type as well.

13

The pain of writing generic code, I guess. You could also use an internal wrapper which implements Deref as .0.borrow(), I suppose.

The same way Borrow isn't a collision, it's an argument instead of an associated type. So the real signature change is

P: Deref + IntoOwned<Target = <P as Deref>::Target>
->
T, P: Borrow<T>, IntoOwned<T>

To justify the loss of a known associated projection type:

  • "Deref coerces to T" is not a Deref<Target = T> bound. Instead, it's more like Deref<Target="Deref coerces to T"> bound; deref coersion is recursive.
  • Borrow is a slight generalization of "deref coerces to T" to all types which "act like T" when used by-ref. std could provide a default impl Borrow<T> for impl Deref<Target = impl Borrow<T>>.
  • IntoOwned is an optimization of p.borrow().to_owned(), not p.to_owned().
  • If you want a concrete type projection, you can always still add a ToOwned bound!

I messed around with min_specialization and splitting into two concepts IntoBorrowed and IntoOwned, but I couldn't find definitions that I was happy with. I agree that IntoOwned feels like a by-value version of ToOwned, and that having such would be theoretically nice; I disagree that's what your provided use case actually wants.

What you want is quite literally P where ∃ &P -> &T, P -> T, with the additional constraint that P is a "smart pointer" of T, here defined as "&P behaves as &T".

14

There's the difference that deref_owned::IntoOwned::Owned (currently) has no bounds, unlike std::borrow::ToOwned::Owned. This seems a bit useless at first, but where needed, additional bounds can be made. Maybe it would be good to add the following to its definition though:

 /// Pointer types that can be converted into an owned type
-pub trait IntoOwned: Sized {
+pub trait IntoOwned: Sized + Deref {
     /// The type the pointer can be converted into
-    type Owned;
+    type Owned: Borrow<<Self as Deref>::Target>;

(akin to ToOwned::Owned having a bound Borrow<Self>)

(Edit: Originally, I named the trait PointerIntoOwned, which kinda justifies the Deref bound.)

With that change, ToOwned and IntoOwned are very similar now.

Maybe Owned<T> should also implement Borrow<T> (just like Cow<'a, T> implements both Deref<Target = T> and Borrow<T>). But …

… having a smart pointer is the basic idea. Hence it must implement Deref (I guess?).

I think Owned is such wrapper, isn't it? Except I don't go through Borrow.

15

I'm still trying to understand this:

Can you explain what you mean, and how such a ToOwned bound could be added for a type projection? ToOwned is more than a type projection. It requires implementing an fn to_owned(&self) -> Self::Owned. I think there are cases when such method can't exist (e.g. for some types that are !Clone and where I can't go from an ordinary reference to an owned value).

But not sure if I'm overlooking all this properly.

I thought my use case was pretty clear, but there seems to be consensus I'm doing things wrong. I don't fully understand yet why or what is wrong with my approach. How is a different solution easier? (Assuming it can be used same easily as Cow but doesn't have the runtime overhead.) And how would it look like exactly?

Maybe I shouldn't have posted here in the first place and first discuss this idea further elsewhere, but when I asked questions like, "shouldn't Rust provide X or do Y differently" on URLO in past, I got referred to IRLO.

To me, Owned is the trivial smart pointer, similiar to Box but without heap allocation. I use it in my library to be generic over returning a re-aligned copy or a reference to some type (but decided at compile-time, and not at run-time for which Cow exists).

Update:

I tried to do that:

-pub trait IntoOwned: Sized {
-    /// The type the pointer can be converted into
-    type Owned;
+pub trait IntoOwned<T> {
     /// Convert into owned type
-    fn into_owned(self) -> Self::Owned;
+    fn into_owned(self) -> T;
 }
 
-impl<'a, T> IntoOwned for &'a T
+impl<'a, T> IntoOwned<<T as ToOwned>::Owned> for &'a T
 where
     T: ?Sized + ToOwned,
 {
-    type Owned = <T as ToOwned>::Owned;
-    fn into_owned(self) -> Self::Owned {
+    fn into_owned(self) -> <T as ToOwned>::Owned {
         self.to_owned()
     }
 }
 
-impl<'a, T> IntoOwned for Cow<'a, T>
+impl<'a, T> IntoOwned<<T as ToOwned>::Owned> for Cow<'a, T>
 where
     T: ?Sized + ToOwned,
 {
-    type Owned = <T as ToOwned>::Owned;
-    fn into_owned(self) -> <Self as IntoOwned>::Owned {
+    fn into_owned(self) -> <T as ToOwned>::Owned {
         Cow::into_owned(self)
     }
 }
 
-impl<T> IntoOwned for Owned<T> {
-    type Owned = T;
-    fn into_owned(self) -> Self::Owned {
-        self.0
+impl<T> IntoOwned<T> for T {
+    fn into_owned(self) -> T {
+        self
     }
 }

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
error[E0119]: conflicting implementations of trait `IntoOwned<&_>` for type `&_`
  --> src/lib.rs:26:1
   |
8  | / impl<'a, T> IntoOwned<<T as ToOwned>::Owned> for &'a T
9  | | where
10 | |     T: ?Sized + ToOwned,
11 | | {
...  |
14 | |     }
15 | | }
   | |_- first implementation here
...
26 |   impl<T> IntoOwned<T> for T {
   |   ^^^^^^^^^^^^^^^^^^^^^^^^^^ conflicting implementation for `&_`

For more information about this error, try `rustc --explain E0119`.
error: could not compile `playground` due to previous error
warning: build failed, waiting for other jobs to finish...
error: could not compile `playground` due to previous error

Note that &'a str has String as owned type, and not str. Thus I did:

impl<'a, T> IntoOwned<<T as ToOwned>::Owned> for &'a T
16

I created a smaller example here:

#![feature(generic_associated_types)]

mod deref_owned {
    use std::borrow::Cow;
    use std::ops::{Deref, DerefMut};

    /// Smart pointer to owned inner value
    #[derive(Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Debug)]
    pub struct Owned<T>(pub T);

    impl<T> Deref for Owned<T> {
        type Target = T;
        /// Returns a reference to the contained value
        fn deref(&self) -> &Self::Target {
            &self.0
        }
    }

    impl<T> DerefMut for Owned<T> {
        /// Returns a mutable reference to the contained value
        fn deref_mut(&mut self) -> &mut Self::Target {
            &mut self.0
        }
    }

    /// Pointer types that can be converted into an owned type
    pub trait IntoOwned: Sized {
        /// The type the pointer can be converted into
        type Owned;
        /// Convert into owned type
        fn into_owned(self) -> Self::Owned;
    }

    impl<'a, T> IntoOwned for &'a T
    where
        T: ?Sized + ToOwned,
    {
        type Owned = <T as ToOwned>::Owned;
        fn into_owned(self) -> Self::Owned {
            self.to_owned()
        }
    }

    impl<'a, T> IntoOwned for Cow<'a, T>
    where
        T: ?Sized + ToOwned,
    {
        type Owned = <T as ToOwned>::Owned;
        fn into_owned(self) -> <Self as IntoOwned>::Owned {
            Cow::into_owned(self)
        }
    }

    impl<T> IntoOwned for Owned<T> {
        type Owned = T;
        fn into_owned(self) -> Self::Owned {
            self.0
        }
    }
}

use deref_owned::*;

use std::fmt::Debug;
use std::ops::Deref;

trait Parse<T: ?Sized> {
    type Pointer<'a>: Deref<Target = T> + IntoOwned + Debug
    where
        Self: 'a;
    fn parse(&self) -> Option<Self::Pointer<'_>>;
}

impl Parse<[u8]> for [u8] {
    type Pointer<'a> = &'a [u8]; // zero-cost
    fn parse(&self) -> Option<Self::Pointer<'_>> {
        Some(self)
    }
}

impl Parse<str> for [u8] {
    type Pointer<'a> = &'a str; // zero-cost
    fn parse(&self) -> Option<Self::Pointer<'_>> {
        std::str::from_utf8(self).ok()
    }
}

impl Parse<i32> for [u8] {
    type Pointer<'a> = Owned<i32>; // smart pointer that drops owned pointee
    /*
    type Pointer<'a> = i32; // won't work
    type Pointer<'a> = std::borrow::Cow<'static, i32>; // comes with runtime overhead
    type Pointer<'a> = Box<i32>; // unnecessary heap allocation
    */
    fn parse(&self) -> Option<Self::Pointer<'_>> {
        Parse::<str>::parse(self)
            .and_then(|s| s.parse::<i32>().ok())
            .map(|i| Owned(i))
            //.map(|i| std::borrow::Cow::Owned(i)) // runtime overhead
            //.map(|i| Box::new(i)) // heap allocation
    }
}

fn foo<T, U>(input: &U)
where
    T: ?Sized,
    U: ?Sized + Parse<T>,
{
    println!(
        "Parsed as {}: {:?}",
        std::any::type_name::<T>(),
        input.parse()
    );
}

fn bar(input: &(impl ?Sized + Parse<str>)) {
    match input.parse() {
        Some(parsed) => println!(
            "{} has {} bytes and {} chars",
            &*parsed,
            parsed.len(),
            parsed.chars().count()
        ),
        None => println!("could not parse"),
    }
}

fn main() {
    let s = b"12345" as &[u8];
    foo::<[u8], _>(s);
    foo::<str, _>(s);
    foo::<i32, _>(s);
    bar(b"H\xC3\xBChnerleiter" as &[u8]);
}

(Playground)

Output:

Parsed as [u8]: Some([49, 50, 51, 52, 53])
Parsed as str: Some("12345")
Parsed as i32: Some(Owned(12345))
Hühnerleiter has 13 bytes and 12 chars

Note how I could use Cow or Box (if I remove the IntoOwned bound, since it's not implemented yet for Box<T> where T: Sized), but each comes with overhead.

Now this is interesting. Look at Box::into_inner, which is unstable. See also tracking issue #80437. It might be another use case for an IntoOwned (or "consuming" ToOwned) trait.

I can't replace Owned<i32> with just i32 in the above example of impl Parse<i32> for [u8]. At least I didn't find a way how to do it.

Beside being Deref, it manages the ownership of the pointee (i.e. the pointer will drop the pointee when the pointer is dropped). That sort of management is one of the aspects a smart pointer is about, I guess.

17

I made a similar project a while back into_owned - Rust

The reason I wanted this is to allow a hashmap entry api to allow passing by owned or by ref, and allowing the caller to chose whether they desire clone to be on the slow path, or whether they have the owned value left over anyway to avoid any clones.

ToOwned is frustrating sometimes since its an abstraction over Clone and not over generic Ownership

2 Likes
18

I don't see a pointer in your example, though. You can impl Deref and DerefMut, but the object itself has nothing to do with pointers.

It seems to me that what you want is a sort of owned pointer to a possibly stack-allocated object. This reminds of the proposals for &move owning pointers. However, as far as I understand, the semantics and use cases of &move are not entirely clear.

19

:sweat_smile: … so I wasn't the first one to come up with that idea! However, I'm still interested in whether it's really (not) possible to use T instead of Owned<T>. It seems like you had the same problem I had. From into_owned::Owned's doc:

Due to implementation conflicts, IntoOwned cannot be directly implemented on T. So instead, this is a light wrapper that allows the implementation.

That's exactly my problem too! :+1:

I see some differences between into_owned and deref_owned:

  • into_owned::Owned doesn't implement Deref
  • into_owned::Owned implements AsRef, Borrow, and From
  • into_owned::IntoOwned uses a type argument instread of an associated type
  • Borrow<T> is a supertrait of into_owned::IntoOwned<T>

This might be due to the different application scenario. For deref_owned::Owned, I consider some other possible trait implementations, like AsMut or BorrowMut. Also, I'm not sure if #derive[Copy] would be useful or harmful. Looking at Box and Cow might give some hints.

Moreover, in case of deref_owned, which implements Deref for Owned<T>, it might be useful to make deref_owned::IntoOwned an associated function in order to avoid shadowing any existing .into_owned() method of T:

pub trait IntoOwned: Sized {
     /// The type the pointer can be converted into
     type Owned;
     /// Convert into owned type
-    fn into_owned(self) -> Self::Owned;
+    fn into_owned(this: Self) -> Self::Owned;
 }

Interestingly, Cow::into_owned is a method on self, while Nightly's Box::into_inner is an associated function :thinking:. Now one (not me) could say that Cow isn't a smart pointer and may thus shadow such a method. But it does implement Deref. :man_shrugging:

Do you have a link to the actual use case (if public)? It might help me to understand the whole issue better (and perhaps also make it easier for me to decide which traits I want to implement for my wrapper when seeing a different potential use case).

Basically Owned<T> is like Box<T>, with a few differences:

  • The memory address where T is stored isn't stable (over the lifetime of the wrapper), i.e. &*some_box as *const _ should be stable even if the box is moved, while &*some_owned_wrapper as *const _ may change (see Playground)
  • Owned<T> can only store T: Sized, while Box<T> works with T: ?Sized.
  • Box always allocates on the heap. Thus if I only want it as a Deref wrapper, I pay a price that could be avoided.

Also Owned<T> is like Cow<'static, U> with <U as ToOwned>::Owned = T without the Cow::Borrowed enum variant.

Thus Owned<T> may have a use case (or two or three?) where neither Box<T> nor Cow<'a, U> would be most efficient. Unless it's possible to fix my use case (and @conradludgate's) in a different manner where we can just use T. But I feel like it's not possible.

That depends on what you mean with "pointer". It doesn't have anything to do with what C would call pointers ("raw pointers" in Rust). As shown above, memory addresses aren't stable either. But when I speak of "smart pointer", I don't talk about the memory model, but instead argue that:

  • Owned<T> can be dereferenced to T
  • Owned<T> manages the memory occupied by T (i.e. it will drop it when the smart pointer is dropped, same as a Box)

In this context, see also this post of above linked "What are smart pointers?" thread, in which @Michael-F-Bryan concludes that Cow may be seen as smart pointer (according to the definition in the same post).

Anyway, I guess it's ultimately a matter of definition. I personally favor the definition of a smart pointer is anything that implements Deref (because it can be dereferenced, thus it "points" to something and isn't a dumb reference). That's of course very different from the concept of pointers in C.

Rust's reference is not very verbose on that matter:

Smart Pointers

The standard library contains additional 'smart pointer' types beyond references and raw pointers.

20

That's a rather useless definition. By your definition, every T is a smart pointer to T, which makes the concept vacuous. Or, if you insist on Deref, any wrapper which implements Deref for method dispatch convenience reasons suddenly becomes a smart pointer. On the contrary, the guidelines are that you should only impl Deref on a smart pointer, so it's not helpful to make this definition recursive.

The concept of smart pointer is pretty well established: it is a pointer (in the C sense, and there is no other meaning) to some data which manages the lifecycle of that data. It's counterproductive to take a well-known concept and assign some entirely different meaning to it. It's like equating "thread-safe" with "thread-local", or saying that [bool] is a string.

&T isn't a smart pointer, it's just a pointer, since it doesn't manage the lifecycle, it just provides some liveness guarantees. T is most definitely not a smart pointer. Box<T> and Rc<T> are smart pointers, they automatically free the resources they contain when all handles go out of scope. Cow<'a, T> is a smart pointer for T=str or T=[U], because in that case the owned variant manages the lifetime of that data. It's not a smart pointer for most other types, e.g. for all T: Sized.

2 Likes