[Pre-RFC] Add a new offset_of macro to core::mem

I meant a reason for the other RFC, `Allow fields in traits that map to lvalues in impl'ing type´. Ideas on disjoint borrowing on custom pointer types (if required at all) fit better in a different thread.

I actually find Swift's solution interesting (kind of like it, but would need to try it out for a verdict) but there are some crucial details. In comparison

  • The input of offset is some well-typed object (a KeyPath from what I could gather) and not a string. It even has a unique expression type for its creation. It overall feels similar to a member pointer (less strictly typed than the current Rust proposal, not every member has a unique type).

  • Secondly, offset is a method of a generic other type which captures the base type in a type parameter and not an ad-hoc intrinsic or macro. This suggests MemoryLayout and KeyPath are the core primitives, not offset.

  • It seems that you are not supposed or at least discouraged to use the integer result of offset for manual computation of say pointers to members. Rather, you can do so type-safely with the KeyPath directly:

    // Mutation through the key path
    root[keyPath: key] = value
    

    Introducing offset alone without similar alternatives/abstractions could be considered rash.

2 Likes

I would very much prefer offset_of!($:type, $:ident), which would allow writing offset_of!(Foo, bar) without quoting bar.

That said, we might want to allow a subset of expression syntax rather than just an ident, to allow offset_of!(Foo, bar.baz) or offset_of!(Foo, bar.baz[3].fnord). Debatable, but people do use the C offsetof that way.

I think as a first step, we should just allow fields, and we can expand to more general expressions later

I would prefer that generality, as it seems more likely to be forward-compatible with disjoint borrows. However I agree with @RustyYato that it need not be the first step, as long as the initial approach does not foreclose such finer resolution as a future extension.

Sounds reasonable to me.

AFAIK, disjointedness only matters with references and accesses, so any API that allows the projection to be done purely as math on raw pointers until the wanted borrow is crated, should be fully capable of allowing disjointed borrows.

2 Likes

Correct. A basic offset_of for fields could be implemented UB-free with a solution for RFC for an operator to take a raw reference by RalfJung · Pull Request #2582 · rust-lang/rfcs · GitHub.

However...

...this does have UB even with the RFC! ptr.field perform an inbounds pointer offset operation, but your pointer is not inbounds of any allocation.

I would like this not to be UB, but there are concerns that making raw pointer field offset a safe operation would lose a lot of optimization potential. Likely, making it UB only on overflow (as opposed to the more restrictive "inbounds") would be sufficient to mitigate that, but LLVM does not offer that option currently.

So, until then, the UB-free way to do offset_of with &raw is:

macro_rules! offset_of {
    ($parent:tt, $field:tt) => {{
        // Make sure the field actually exists. This line ensures that a
        // compile-time error is generated if $field is accessed through a
        // Deref impl.
        let $parent { $field: _, .. };

        // Create an instance of the container and calculate the offset to its field.
        // Here we're using an uninitialized instance of $parent. We avoid UB
        // by only using raw pointers that point to real (allocated, albeit uninitialized) memory.
        let val = $crate::mem::MaybeUninit::<$parent>::uninit();
        let base_ptr = val.as_ptr();
        #[allow(unused_unsafe)] // for when the macro is used in an unsafe block
        let field_ptr = unsafe { &raw (*base_ptr).$field };
        let offset = (field_ptr as usize) - (base_ptr as usize);
        offset
    }};
}
5 Likes

Follow-up: the actual reason this has UB is that dereferencing dangling pointers is UB. So It's the *ptr in your code that already causes UB.

At some point I want to pursue relaxing this for raw pointers, but there are more pressing matters. :wink:

1 Like

Now that offset_from is a const fn (behind a feature flag), this can be implemented as a const-friendly macro (I haven't tested this since I nightly hasn't yet been updated with const_ptr_offset_from; I may revise this in the next day or two when I get a chance to actually test it):

#![feature(const_transmute)]
#![feature(const_ptr_offset_from)]
#![feature(ptr_offset_from)]

macro_rules! offset_of {
    ($Struct:path, $($field:tt)+) => ({
        const OFFSET: usize = {
            extern crate core;

            let base_uninit = core::mem::MaybeUninit::<$Struct>::uninit();
            unsafe {
                // This is UB, but it's the best we can do for the time being.
                // If this is in `core`, we can just leave a comment to tell
                // people that this is an unsafe implementation detail that
                // works here in `core` but is UB outside of this macro.
                let base_ref = core::mem::transmute::<_, &$Struct>(&base_uninit);
                let base_u8_ptr = base_ref as *const _ as *const u8;

                let field_u8_ptr = &base_ref.$($field)+ as *const _ as *const u8;
                let offset = field_u8_ptr.offset_from(base_u8_ptr) as usize;
                
                // Make sure the offset computation stayed within the struct's size.
                let assert = [offset; 1];
                assert[(offset <= core::mem::size_of::<$Struct>()) as usize - 1]
            }
        };
        OFFSET
    })
}

Using $($field:tt)+ allows this to support offset_of!(Struct, field.sub_field.sub_array[3]).

You had to drop the deref protection for this, though.

If a deref coercion actually ends up in a different allocation, offset_from will detect that -- you crucially rely on this computation happening at compile-time though, so there should be a comment for that.

But it is also conceivable that a deref coercion stays inside the same struct... and I am curious how weird that can get? I first thought this could be impure, but since you are running this at const-time we'd actually get an error if the deref code wasn't const-compatible. Probably right now being in a const context entirely protects you against deref coercions as those are non-const fn-calls; that check will get relaxed eventually though.

Yeah, I thought about that. But the deref protection syntax isn't compatible with tuple structs, so I consider it non-viable unfortunately.

Then you should mark the macro as unsafe though.

Actually it works just fine with tuple structs.

4 Likes

Indeed it does. I must have screwed something up when I tried it earlier. I stand corrected. Thanks.

This topic was automatically closed 540 days after the last reply. New replies are no longer allowed.