Idea: `Thin` wrapper of DSTs

language design
1

Given that:

  • List<T> in rustc that is a thin [T] with the metadata (length) on the head;
  • ThinVec<T> that put the length and capacity components together with its contents on the heap;
  • ThinBox<T> like Box<T> but put the metadata together on the heap;
  • thin_trait_object, an attribute macro that makes a thin trait object (by manually constructing the vtable).

Could we have a Thin<T> like this:

/// A wrapper that wraps a `T: Pointee` together with
/// its metadata that makes the type "thin":
pub struct Thin<T: Pointee> {
    metadata: T::Metadata,
    data: EraseMetadata<T>,
}

// We can know its size from its metadata.
impl<T: Pointee + MetaSized> MetaSized for Thin<T> {}

/// A wrapper that ignores the metadata of a type.
#[lang = "erase_metadata"]
pub struct EraseMetadata<T: Pointee>(T);

// The size is unknown because the metadata is erased.
impl<T: Pointee> PointeeSized for EraseMetadata<T> {}

Then List<T>, ThinVec<T>, and ThinBox<T> could be (in representation) Thin<[T]>, Box<(usize, Thin<[MaybeUninit<T>]>)>, and Box<Thin<T>>.

Moreover, it makes DSTs like Thin<dyn Trait> FFI-compatible (could be passed by a pointer).

3 Likes
[Pre-RFC] Thin pointers with inlined metadata
2

I wonder how you would create an instance of Thin/ErasedMetadata though.

ThinVec doesn't store the capacity inline.

3

I referred to this implementation of Header where the ThinVec points to. It does contain the capacity inline.

By unsizing from a sized value, i.e. impl<T: Unsize<U>, U: PointeeSized> Unsize<Thin<U>> for T {} (probably implemented by the compiler, then Thin must also be a lang item)

4

The capacity is inline in the Header struct, but the Header itself is stored on the heap.

5

There was a typo, (usize, Thin<[MaybeUninit<T>]>) should be Box<(usize, Thin<[MaybeUninit<T>]>)>.

1 Like
6

Hmm, Thin would be a weird new thing: it's unsized but has no metadata. This unfortunately breaks the invariant that <T as Pointee>::Metadata=() implies T: Sized. I don't know if this can cause problems in practice.

So to manipulate it you may need to query e.g. its size, but instead of looking at the pointer as is usual for unsized types, you have to dereference the pointer. That feels like an important thing to have honestly, having to do that by hand seems tricky and likely doesn't extend to things like CoercePointee.

Also if we can have that then we can also have unsized enums! I.e. enums where we only allocate enough space for the current variant instead of the biggest variant. That would use the same mechanism.

I'd quite like this wrapper honestly

7

Is it possible to define Thin<T> like an extern type?

EDIT: I mean, having Thin<T> not impl Pointee, so it doesn't break <T as Pointee>::Metadata=() implies T: Sized.

Yes, that's different from the usual unsized types.

8

I re-read the sized_hierarchy RFC (https://github.com/rust-lang/rfcs/pull/3729) and actually I don't think Metadata=()=>Sized holds. Indeed as you say for extern types that's not the case, so we could make it PointeeSized and not MetaSized.

I wonder how to get good ergonomics for this type though. E.g. std::ptr::metadata has to return (), so we'd need something else to fetch the real metadata. It breaks the fairly deep assumption that "if a value needs metadata then that metadata is stored in any pointer to it".

1 Like
9

Ah yes, I see ValueSized proposed at the end of that RFC, for types that can have their metadata computed from their value. That's what Thin needs.

1 Like
10

How about providing Thin::metadata to fetch the metadata of a Thin pointer? std::ptr::metadata is safe and takes a raw pointer *const T, which doesn't fit Thin that requires a dereferencable pointer.

impl<T: Pointee> Thin<T> {
    pub fn metadata(this: &Self) -> T::Metadata {
        this.metadata
    }
}
1 Like
11

Take a second look, this question seems not so trivial. Creating a Thin on heap will not be a problem (just similar to ThinBox), but creating it on stack is more complicated.

Taking Thin<[u8]> for example:

  • first, construct a sized value on stack, like let value = [0_u8; 4] (with type [u8; 4]),
  • second, obtain the metadata (e.g. by let metadata = std::ptr::metadata(&value as &[u8])),
  • third, combine the metadata together with the value, i.e. let thin = Thin { metadata, data: value }.
    • The problem is: what's the type of thin?
      • Thin<[u8; 4]>? No, <[u8; 4] as Pointee>::Metadata is () instead of usize;
      • Thin<[u8]>? No, it's unsized so cannot be constructed on stack.

The solution I can think of is to provide a builtin-macro thin!(), in which thin!([0_u8; 4] as [u8]) returns a reference to the unsized value of &mut Thin<[u8]>. (i.e., let the compiler handle the intermediate sized Thin value with the unsized metadata.)

12

The thing stopping on-stack slices from existing is not the metadata, so Thin doesn't have to solve that problem either. If let val: [u8] = … is ever supported, presumably Thin<[u8]> could be supported too with the same magic.

Having {Box, Rc, Arc}::new_thin(&) would be useful anyway.

&[u8; N] can't be coerced to &Thin<[u8]> due to lack of space for the metadata, but something like this could:

struct NotYetCoercedThin<T: Sized, U: ?Sized> where T: Unsize<U> {
     metadata: U::Metadata,
     data: T,
}

You'd create NotYetCoercedThin<[u8; N], [u8]> on the stack (or some other place) and then coerce or transmute &NotYetCoercedThin<[u8; N], [u8]> to &Thin<[u8]>.

1 Like
13

How about let Thin takes 2 generic parameters instead?

pub struct Thin<T: Pointee, U: Pointee = T> {
    metadata: U::Metadata,
    data: EraseMetadata<T>,
}

Then we could have

// in the compiler
impl<T: Pointee, U: Pointee> Unsize<Thin<U>> for Thin<T, U>
where
    T: Unsize<U>,
{}

impl<T: Sized> Thin<T> {
    pub fn new(value: T) -> Self { /* ... */ }
    pub fn with_metadata<U: Pointee>(self) -> Thin<T, U>
    where
        T: Unsize<U>,
    { /* ... */ }
}

fn foo() {
    let sized: [u8; 1] = [0_u8];
    let thin_sized: Thin<[u8; 1]> = Thin::new(sized);
    let thin_sized_with_metadata: Thin<[u8; 1], [u8]> = thin_sized.with_metadata();
    let ref_thin_sized: &Thin<[u8; 1], [u8]> = &thin_sized_with_metadata;
    let ref_thin_unsized: &Thin<[u8]> = ref_thin_sized; // coerce unsized
}
14

It's necessary to have one-type-arg version to be able to abstract types, such as Thin<dyn Fn>.

15

Wouldn't Thin<dyn Foo, dyn Foo> work for that with this definition?

17

I guess it could, if the Unsize/EraseMetadata trait had a blanket identity impl.

But you'd still need a type with a sized type arg and coercions/casts to construct unsized one (especially if you want to support placing inlined-dyn objects deep inside other structs, not just as a standalone thin heap box or let binding).

18

I opened a pre-RFC here: [Pre-RFC] Thin pointers with inlined metadata