[pre-RFC] Trait in std::clone for cloning to a different lifespan

[jmillikin]

Recently I’ve been working on a library where user code gets called with borrowed references to underlying buffers. Avoiding memcpy is important in the common case, but sometimes the user will want to spawn off a thread to process the request async. The definition of Clone makes that challenging because the lifetimes are part of the cloned type, so there’s no way to define fn clone(Self<'a>) -> Self<'static>.

fn handler<'a>(req: Cow<'a, [u8]>) {
  let req = req.clone();
  thread::spawn(move || { /* fails, `req` still in 'a */ });
}

It seems like having a CloneTo capable of cloning to a different type (and thus different lifetime) would be useful here. There’s precedent in std::borrow::ToOwned, which exists for cross-type borrowing. The definition is straightforward:

// std::clone::CloneTo
trait CloneTo<Target> {
    fn clone_to(&self) -> Target;
}

With this, I can freely clone Cow and other existentially quantified types to other lifetimes (including 'static):

impl <'from, 'to, B> CloneTo<Cow<'to, B>> for Cow<'from, B>
    where B: ToOwned + ?Sized + 'to
{
    fn clone_to(&self) -> Cow<'to, B> {
        match *self {
            Cow::Borrowed(b) => Cow::Owned(b.to_owned()),
            Cow::Owned(ref o) => {
                let b: &B = o.borrow();
                Cow::Owned(b.to_owned())
            },
        }
    }
}

#[derive(Debug)]
struct Foo<'a> {
    s: Cow<'a, str>,
}

impl<'from, 'to> CloneTo<Foo<'to>> for Foo<'from> {
    fn clone_to(&self) -> Foo<'to> {
        Foo { s: self.s.clone_to() }
    }
}

Does this seem useful enough to justify an RFC?

[Centril]

I tried checking if we could retrofit the existing Clone trait for something like this…

#![feature(arbitrary_self_types)]

fn main() {}

use std::ops::Deref;

pub trait Clone<Target = Self>
where
    Target: Deref<Target = Self>
{
    fn clone(&self) -> Target;
    fn clone_from(self: &mut Target, source: &Self) {
        *self = source.clone();
    }
}

struct Foo;
impl Clone for Foo {
//   ^^^^^ error[E0277]: the trait bound `Foo: std::ops::Deref` is not satisfied
    fn clone(&self) -> Target {
        Foo
    }
}

How unsatisfying.

[drXor]

Is there any reason we can’t add

default impl<T> Deref for T {
    type Target = Self;
    fn deref(&self) -> &Self::Target { self }
}

with coherence? We can probably just add a special rule to autoderef rules and *x to ignore this impl.

[Centril]

Is there any reason we can’t add

That impl doesn’t actually provide an implementation; it is just a blueprint that you can use to avoid filling in the items. (See the specialization RFC).

Barring other unforeseen problems, what you would need is specialization to do this:

impl<T> Deref for T {
    default type Target = Self;
    default fn deref(&self) -> &Self::Target { self }
}

[drXor]

Ok, it’s pretty clear I have no idea how specialization works…

Would the snippet you wrote down DTRT, then?

[Centril]

Would the snippet you wrote down DTRT, then?

Unfortunately not, if you try to write:

#![feature(specialization)]

fn main() {}

pub trait Deref {
    type Target: ?Sized;
    fn deref(&self) -> &Self::Target;
}

impl<T> Deref for T {
    default type Target = Self;
    default fn deref(&self) -> &Self::Target { &*self }
}

you will get the error:

error[E0308]: mismatched types
  --> src/main.rs:12:48
   |
12 |     default fn deref(&self) -> &Self::Target { &*self }
   |                                                ^^^^^^ expected associated type, found type parameter
   |
   = note: expected type `&<T as Deref>::Target`
              found type `&T`

error: aborting due to previous error

This occurs because specialization does not allow you to assume that Target and deref have not been specialized together. You would need some sort of “specialization groups” to do it I think:

impl<T> Deref for T {
    default {
        type Target = Self;
        fn deref(&self) -> &Self::Target { &*self }
    }
}

[dhardy]

I was going to ask if this is the same as reborrowing, but no, it’s not.

But given your problem, can’t you just call Cow::into_owned?

[jmillikin]

But given your problem, can’t you just call Cow::into_owned ?

Cow was used as an example of a type with a lifetime. The proposed trait would ideally be useful for types that aren’t conceptually smart pointers.

Here’s a more complex example, where clone_to() is used to copy types that have an internal Cow (Response), and types with more complex clone logic (cloning a response callback needs to convert &'a mut File to Arc<Mutex<File>>).

fn getxattr<'a>(
    &self,
    ctx: &fuse::ServerContext<'a>,
    request: &fuse::GetxattrRequest<'a>,
    respond: Respond<'a, fuse::GetxattrResponse<'a>>,
) {
    if request.name().to_bytes() == b"user.checksum.sha256" {
        let path = self.node_cache.path(ctx.node_id());
        let resp = fuse::GetxattrResponse::new(request);
        let respond = respond.clone_to();
        thread::spawn(move || {
            let checksum = sha256(path);
            let resp = resp;
            resp.set_value(&checksum);
            respond.ok(resp);
        });
    }
    respond.err(io::Error::from_raw_os_error(libc::ENODATA));
}

[dhardy]

If I understand right, respond contains a pointer that you wish to re-borrow with a sub-lifetime (i.e. which must exist no longer than the given 'a)? If so, see the issue I linked about reborrowing. Otherwise, do you wish to fully clone an item to avoid lifetime restrictions? (I don’t think this is what you’re trying to do, but you can’t simply cast a 'a to a 'static.)

[jmillikin]

Otherwise, do you wish to fully clone an item to avoid lifetime restrictions? (I don’t think this is what you’re trying to do, but you can’t simply cast a 'a to a 'static .)

Yes, that is exactly what I want to do. Given a value with a constrained lifetime (because it contains references), I want to clone it to a new value of the same type with 'static lifetime. The cloned object can then be passed to a thread for async processing.

[rpjohnst]

This doesn’t really work in general. Most of the time when a type has a lifetime parameter, it’s because it contains a reference. Doing a deep copy to get rid of the lifetime makes sense conceptually, but then who would own the new object that the reference points to? To make it 'static, it would have to leak.

You could just use ToOwned directly- define the Owned associated type as the 'static version of the same type for these Cow-based types, since they’re the only ones where this operation can be done without leaking anyway.

[jmillikin]

This doesn’t really work in general. Most of the time when a type has a lifetime parameter, it’s because it contains a reference. Doing a deep copy to get rid of the lifetime makes sense conceptually, but then who would own the new object that the reference points to? To make it 'static , it would have to leak.

The deeply cloned object wouldn’t contain a reference, it would contain an owned value. There’s no leaks involved here.

You could just use ToOwned directly- define the Owned associated type as the 'static version of the same type for these Cow -based types, since they’re the only ones where this operation can be done without leaking anyway.

This isn’t possible because there’s a blanket impl<T: Clone> ToOwned T.

[CAD97]

I think what you really want is an IntoOwned:

trait IntoOwned {
    type Owned: 'static;
    fn into_owned(self) -> Self::Owned;
}

// Yes I know this doesn't actually work since `Self::Owned`
// and `Self::into_owned` are specialized separately.
// Plus you can't specialize over lifetimes, right?
default impl<T: 'static> IntoOwned for T {
    type Owned = Self;
    fn into_owned(self) -> Self { self } 
}

impl<'a, T: Clone> IntoOwned for Cow<'a, T> {
    type Owned = Cow<'static, T>;
    fn into_owned(self) -> Self::Owned {
        Cow::Owned(Cow::into_owned(self))
    }
}

The problem here requiring a solution along these lines is that Clone::cloneing (and thus ToOwned::to_owneding) a Cow (thus any type containing a Cow that derives Clone) makes the value owned and thus an owned 'static value, but the lifetime is still there because the Cow clone impl doesn’t relax the Cow lifetime (or that of any type containing it, of course).

Note that using into_owned is also effectively free in the already-owned usecase, unlike the proposed clone_to, since that requires cloning self in all cases.

[rpjohnst]

The deeply cloned object wouldn’t contain a reference, it would contain an owned value. There’s no leaks involved here.

Right, I’m talking about the general case, which is why my next paragraph says “Cow-based types … are the only ones where this operation can be done without leaking anyway.”

The blanket impl is unfortunate if you also need Clone, but my point is that you don’t want CloneTo with an arbitrary output lifetime because that doesn’t make sense.

[jmillikin]

I think what you really want is an IntoOwned :

This consumes the value instead of cloning it, which is specifically not what I want because the handler doesn’t own the original value. It’s only got a lifetime-delimited reference, which I would like to clone into a non-reference value with the <'a> lifetime type parameter replaced by <'static>.

Right, I’m talking about the general case, which is why my next paragraph says “ Cow -based types … are the only ones where this operation can be done without leaking anyway.”

I don’t understand what this could mean. There isn’t anything special about Cow, right? It’s just a standard ADT – you could imagine doing a similar deep clone for any type with the semantics “may (not must) contain a reference”.

The blanket impl is unfortunate if you also need Clone , but my point is that you don’t want CloneTo with an arbitrary output lifetime because that doesn’t make sense.

I do want it with an arbitrary output lifetime, because that’s the only way to get <'static>. A 'static lifetime can implicitly decay into any other lifetime, so if you can clone to 'static then you can clone to anything.

[CAD97]

This consumes the value instead of cloning it, which is specifically not what I want because the handler doesn’t own the original value. It’s only got a lifetime-delimited reference, which I would like to clone into a non-reference value with the <'a> lifetime type parameter replaced by <'static> .

But in your more complex example:

fn getxattr<'a>(
    &self,
    ctx: &fuse::ServerContext<'a>,
    request: &fuse::GetxattrRequest<'a>,
    respond: Respond<'a, fuse::GetxattrResponse<'a>>,
) {
    if request.name().to_bytes() == b"user.checksum.sha256" {
        let path = self.node_cache.path(ctx.node_id());
        let resp = fuse::GetxattrResponse::new(request);
        let respond = respond.clone_to();
        thread::spawn(move || {
            let checksum = sha256(path);
            let resp = resp;
            resp.set_value(&checksum);
            respond.ok(resp);
        });
    }
    respond.err(io::Error::from_raw_os_error(libc::ENODATA));
}

The only cloned value here is respond: Respond<'a, fuse::GetxattrResponse<'a>, which is an owned value of type Respond<'_, _>. In any case, you could just impl IntoOwned for &_ anyway:

impl<'a, T: Clone + IntoOwned> IntoOwned for &'a T {
    type Owned = <T as IntoOwned>::Owned;
    fn into_owned(self) -> T {
         <T as IntoOwned>::into_owned(<T as Clone>::clone(self))
    }
}

[rpjohnst]

I do want it with an arbitrary output lifetime, because that’s the only way to get <'static> . A 'static lifetime can implicitly decay into any other lifetime, so if you can clone to 'static then you can clone to anything.

The point is that 'static is the only output lifetime that gives you any new capabilities. (And yes, you’re right- “Cow-based” includes arbitrary sum types that can leave out the reference when necessary.)

[jmillikin]

The only cloned value here is respond: Respond<'a, fuse::GetxattrResponse<'a> , which is an owned value of type Respond<'_, _> . In any case, you could just impl IntoOwned for &_ anyway:

The request and context should also be clone-to-able here – I could in theory make them non-reference values, but that seems to confuse the optimizer into inserting a memcpy().

I suppose the trait could be called CloneOwned or something like that. The cloning is important to call out, because it’s going to be slow.

The point is that 'static is the only output lifetime that gives you any new capabilities.

But it’s not actually the 'static part that’s important. Semantically the operation is “clone to other lifetime”, which happens to be 'static in the special case of thread spawning. You could imagine using this with Crossbeam or another scoped thread crate, in which case it’d be cloning to the thread pool’s lifetime.

The type conversion is logically something like clone_to<T, 'a, 'b>(&(T + 'a)) -> (T + 'b).

[CAD97]

The point is that 'static is the only output lifetime that gives you any new capabilities.

But it’s not actually the 'static part that’s important.

I think rpjohnst’s point here is that it’d be much simpler just to fix the output lifetime to 'static to make clearer the operation here (which is to convert from a potentially borrowed Cow-like structure to an owned value without changing the type). Then if you need a shorter lifetime, that 'static lifetime can be reborrowed or decay to the shorter lifetime.

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.