I have a potentially unsized type Record defined as follows:
#[repr(C)]
struct Record<H, T: ?Sized> {
header: H,
data: T
}
These are allocated (boxed) using concrete (i.e. sized) types, but only pointers to the data field are ever handed out to users. By the time a record needs to be de-allocated, the handed out pointer may or may not have lost it's concrete type information, i.e. *mut T may be a fat pointer.
Given that Record is #[repr(C)] and has only two members, it is fairly ease to calculate the offset from the (unsized) data field to the header aka the top of the struct:
impl<H, T: ?sized> Record<H, T> {
unsafe fn header_from_data(data: *mut T) -> *mut H {
// align_of_val should really just accept a pointer but alas...
let data_align = mem::align_of_val(&*data);
// if data is a fat pointer, the cast to u8 turns it into a thin one
(data as *mut u8).sub(Self::data_offset(data_align)).cast()
}
const fn data_offset(data_align: usize) -> usize {
// matches the layout algorithm used by `rustc` for C-like structs
let offset = mem::size_of::<H>();
offset + offset.wrapping_neg() % data_align
}
}
However, in order to de-allocate a Record again, it would be necessary to cast the *mut H into a *mut Record<T, H>, but this is not possible, since the latter is potentially a fat pointer. However, the required metadata (or vtable pointer) is already present in the (maybe fat) *mut T pointer. Unfortunately, all pointer arithmetic operations defined on pointer types like add/sub or offset only work with count differentials instead of bytes, and the usual approach of casting to a pointer to a type with size 1 like u8 isn't applicable here, since this cast would discard the vtable and make the pointer thin again. Interestingly, I can't even use the unstable TraitObject type here, since the passed *mut T is not necessarily a fat pointer.
I ended up writing the following function:
unsafe fn record_from_data(data: *mut T) -> *mut Self {
// pointer is cast to a pointer to an unsized `Record<H, dyn ..>`, but
// in fact still points at the record's `data` field
let mut ptr = data as *mut Self;
// header is the "correct" thin pointer, since the header field is at
// offset 0
let header = Self::header_from_data(data) as *mut ();
{
// create a pointer to the fat pointer's "data" half
// this is the dangerous part, since the layout of fat pointers to
// trait objects is not actually guaranteed in any way, but is
// currently implemented this way
let data_ptr = &mut ptr as *mut *mut Self as *mut *mut ();
*data_ptr = header;
}
ptr
}
Since the layout of fat pointers is technically not specified and may change, this solution is of course potentially brittle and perhaps even UB, although I am not sure, since it is in fact how it is currently (internally) implemented and I somehow don't think the pointer part of a fat pointer will ever not be the first field of a fat pointer.
I am fairly sure there is nothing in the language currently that would solve my problem in a way that doesn't make assumptions about the layout of fat pointers, so I was wondering how the language or standard library could be amended to solve this issue. One solution would be to add functions that allow byte-wise pointer arithmetic on pointers to types of arbitrary sizes. Another could be function like this:
impl<T: ?Sized> *mut T {
pub fn as_thin_ptr(&mut self) -> &mut *mut () { /* ... */}
}
Although is probably too specialized with not enough general use to warrant adoption into the standard library.